{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,10]],"date-time":"2026-06-10T20:58:15Z","timestamp":1781125095786,"version":"3.54.1"},"reference-count":46,"publisher":"American Society of Clinical Oncology (ASCO)","issue":"2","content-domain":{"domain":["ascopubs.org"],"crossmark-restriction":true},"short-container-title":["JCO Clin Cancer Inform"],"published-print":{"date-parts":[[2026,4]]},"abstract":"<jats:sec>\n                    <jats:title>PURPOSE<\/jats:title>\n                    <jats:p>Multimodal machine learning offers a holistic view of a patient's status, integrating structured and unstructured data from electronic health records (EHR). We propose a framework to predict metastasis risk 1 month before diagnosis, using 6 months of clinical history from EHR data.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>METHODS<\/jats:title>\n                    <jats:p>Data from four cancer cohorts collected at Karolinska University Hospital (Stockholm, Sweden) were analyzed: breast (n = 743), colon (n = 387), lung (n = 870), and prostate (n = 1,890). The data set included demographics, comorbidities, laboratory results, medications, and clinical text. We compared traditional and deep learning (DL) classifiers across single modalities and multimodal combinations, using various fusion strategies and a transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) 2a design, with an 80-20 development-validation split to ensure a rigorous, repeatable evaluation. Performance was evaluated using AUROC, area under the precision-recall curve, F1 score, sensitivity, and specificity. We then employed a multimodal adaptation of Shapley additive explanations (SHAP) to analyze the classifiers' reasoning.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>RESULTS<\/jats:title>\n                    <jats:p>Intermediate fusion achieved the highest F1 scores on breast (0.845), colon (0.786), and prostate cancer (0.845), demonstrating strong predictive performance. For lung cancer, the intermediate fusion achieved an F1 score of 0.819, while the text-only model achieved the highest, with an F1 score of 0.829. DL classifiers consistently outperformed traditional models. Colon cancer, the smallest cohort, had the lowest performance, highlighting the importance of sufficient training data. SHAP analysis showed that the relative importance of modalities varied across cancer types.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>CONCLUSION<\/jats:title>\n                    <jats:p>Fusion strategies offer distinct strengths and weaknesses. Intermediate fusion consistently delivered the best results, but strategy choices should align with data characteristics and organizational needs.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1200\/cci-25-00141","type":"journal-article","created":{"date-parts":[[2026,6,10]],"date-time":"2026-06-10T20:00:18Z","timestamp":1781121618000},"update-policy":"https:\/\/doi.org\/10.1200\/crossmark","source":"Crossref","is-referenced-by-count":0,"title":["Multimodal Machine Learning for Early Prediction of Metastasis in a Swedish Multicancer Cohort"],"prefix":"10.1200","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7693-0576","authenticated-orcid":true,"given":"Franco","family":"Rugolon","sequence":"first","affiliation":[{"name":"Department of Computer and Systems Sciences, Stockholm University, Stockholm, Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7938-2747","authenticated-orcid":true,"given":"Korbinian","family":"Randl","sequence":"additional","affiliation":[{"name":"Department of Computer and Systems Sciences, Stockholm University, Stockholm, Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-3500-0576","authenticated-orcid":true,"given":"Braslav","family":"Jovanovic","sequence":"additional","affiliation":[{"name":"Tema Cancer, Karolinska University Hospital, Stockholm, Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1357-1967","authenticated-orcid":true,"given":"Ioanna","family":"Miliou","sequence":"additional","affiliation":[{"name":"Department of Computer and Systems Sciences, Stockholm University, Stockholm, Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4632-4815","authenticated-orcid":true,"given":"Panagiotis","family":"Papapetrou","sequence":"additional","affiliation":[{"name":"Department of Computer and Systems Sciences, Stockholm University, Stockholm, Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"233","reference":[{"key":"e_1_3_6_2_2","doi-asserted-by":"publisher","DOI":"10.3322\/caac.21871"},{"key":"e_1_3_6_3_2","volume-title":"Global Cancer Observatory: Cancer Tomorrow","author":"Ferlay J","year":"2024","unstructured":"Ferlay J, Laversanne M, Ervik M, et al: Global Cancer Observatory: Cancer Tomorrow, 2024. https:\/\/gco.iarc.who.int\/tomorrow"},{"key":"e_1_3_6_4_2","doi-asserted-by":"publisher","DOI":"10.3322\/caac.21820"},{"key":"e_1_3_6_5_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ejca.2005.01.028"},{"key":"e_1_3_6_6_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cllc.2014.05.006"},{"key":"e_1_3_6_7_2","doi-asserted-by":"publisher","DOI":"10.1007\/s12282-011-0276-3"},{"key":"e_1_3_6_8_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10555-016-9618-0"},{"key":"e_1_3_6_9_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.csbj.2014.11.005"},{"key":"e_1_3_6_10_2","article-title":"Enhancing case capture, quality, and completeness of primary melanoma pathology records via natural language processing","author":"Malke JC","unstructured":"Malke JC, Jin S, Camp SP, et al: Enhancing case capture, quality, and completeness of primary melanoma pathology records via natural language processing. JCO Clin Cancer Inform 10.1200\/CCI.19.00006","journal-title":"JCO Clin Cancer Inform"},{"key":"e_1_3_6_11_2","doi-asserted-by":"publisher","DOI":"10.3414\/ME10-01-0038"},{"key":"e_1_3_6_12_2","article-title":"Development and evaluation of perl-based algorithms to classify neoplasms from pathology records in synoptic report format","author":"Rossi KR","unstructured":"Rossi KR, Echeverria D, Carroll A, et al: Development and evaluation of perl-based algorithms to classify neoplasms from pathology records in synoptic report format. JCO Clin Cancer Inform 10.1200\/CCI.20.00152","journal-title":"JCO Clin Cancer Inform"},{"key":"e_1_3_6_13_2","doi-asserted-by":"publisher","DOI":"10.1002\/pds.5309"},{"key":"e_1_3_6_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2023.3243854"},{"key":"e_1_3_6_15_2","doi-asserted-by":"publisher","DOI":"10.1186\/s12967-025-06686-x"},{"key":"e_1_3_6_16_2","doi-asserted-by":"publisher","DOI":"10.1093\/bib\/bbac448"},{"key":"e_1_3_6_17_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.csbj.2021.12.028"},{"key":"e_1_3_6_18_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-025-98188-5"},{"key":"e_1_3_6_19_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.clinimag.2024.110254"},{"key":"e_1_3_6_20_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00432-023-05574-5"},{"key":"e_1_3_6_21_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00259-024-07065-2"},{"key":"e_1_3_6_22_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-025-09691-8"},{"key":"e_1_3_6_23_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.compedu.2005.05.005"},{"key":"e_1_3_6_24_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41698-024-00649-z"},{"key":"e_1_3_6_25_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-023-42811-4"},{"key":"e_1_3_6_26_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.crad.2025.106832"},{"key":"e_1_3_6_27_2","doi-asserted-by":"publisher","DOI":"10.34133\/2022\/9860179"},{"key":"e_1_3_6_28_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11548-022-02698-w"},{"key":"e_1_3_6_29_2","first-page":"87","volume-title":"Joint European Conference on Machine Learning and Knowledge Discovery in Databases","author":"Rugolon F","year":"2023","unstructured":"Rugolon F, Randl K, Bampa M, et al: A workflow for creating multimodal machine learning models for metastasis predictions in melanoma patients. Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Cham, Switzerland: Springer 87-102, 2023"},{"key":"e_1_3_6_30_2","first-page":"1","article-title":"HEALTH BANK-A workbench for data science applications in healthcare","volume":"1381","author":"Dalianis H","year":"2015","unstructured":"Dalianis H, Henriksson A, Kvist M, et al: HEALTH BANK-A workbench for data science applications in healthcare. CAiSE Ind Track 1381:1-18, 2015","journal-title":"CAiSE Ind Track"},{"key":"e_1_3_6_31_2","volume-title":"ICD-10 Version: 2025\u2014icd.who.int","author":"WHO","year":"2024","unstructured":"WHO: ICD-10 Version: 2025\u2014icd.who.int. 2024. https:\/\/www.icd10data.com\/ICD10CM\/Codes\/C00-D49\/"},{"key":"e_1_3_6_32_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ejca.2022.07.025"},{"key":"e_1_3_6_33_2","doi-asserted-by":"publisher","DOI":"10.63317\/3g44z99gre9a"},{"key":"e_1_3_6_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.1967.1053964"},{"key":"e_1_3_6_35_2","doi-asserted-by":"publisher","DOI":"10.1002\/9781118548387"},{"key":"e_1_3_6_36_2","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939785"},{"key":"e_1_3_6_37_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10618-019-00647-x"},{"key":"e_1_3_6_38_2","doi-asserted-by":"publisher","unstructured":"Devlin J: Bert: Pre-training of deep bidirectional transformers for language understanding. ArXiv Prepr ArXiv181004805 10.48550\/arXiv.1810.04805","DOI":"10.48550\/arXiv.1810.04805"},{"key":"e_1_3_6_39_2","doi-asserted-by":"publisher","DOI":"10.7326\/M14-0698"},{"key":"e_1_3_6_40_2","doi-asserted-by":"publisher","DOI":"10.1111\/j.2517-6161.1974.tb00994.x"},{"key":"e_1_3_6_41_2","unstructured":"Cawley GC Talbot NLC: On Over-fitting in Model Selection and Subsequent Selection Bias in Performance Evaluation 2010"},{"key":"e_1_3_6_42_2","first-page":"2825","article-title":"Scikit-learn: Machine learning in python","volume":"12","author":"Pedregosa F","year":"2011","unstructured":"Pedregosa F, Varoquaux G, Gramfort A, et al: Scikit-learn: Machine learning in python. J Mach Learn Res 12:2825-2830, 2011","journal-title":"J Mach Learn Res"},{"key":"e_1_3_6_43_2","doi-asserted-by":"publisher","unstructured":"L\u00f6ning M Bagnall A Ganesh S et al: Sktime: A Unified Interface for Machine Learning with Time Series. ArXiv Prepr ArXiv190907872. 10.48550\/arXiv.1909.07872","DOI":"10.48550\/arXiv.1909.07872"},{"key":"e_1_3_6_44_2","volume-title":"Deep Learning with Python","author":"Chollet F","year":"2021","unstructured":"Chollet F. Deep Learning with Python. New York, NY: Simon and Schuster, 2021"},{"key":"e_1_3_6_45_2","article-title":"Pytorch: An imperative style, high-performance deep learning library","volume":"32","author":"Paszke A","year":"2019","unstructured":"Paszke A, Gross S, Massa F, et al: Pytorch: An imperative style, high-performance deep learning library. Adv Neural Inf Process Syst 32, 2019","journal-title":"Adv Neural Inf Process Syst"},{"key":"e_1_3_6_46_2","first-page":"4765","volume-title":"Advances in Neural Information Processing Systems 30","author":"Lundberg SM","year":"2017","unstructured":"Lundberg SM, Lee SI. A unified approach to interpreting model predictions. In: Guyon I, Luxburg UV, Bengio S, et al, eds. Advances in Neural Information Processing Systems 30. Red Hook, NY: Curran Associates; 4765-4774, 2017. http:\/\/papers.nips.cc\/paper\/7062-a-unified-approach-to-interpreting-model-predictions.pdf"},{"key":"e_1_3_6_47_2","doi-asserted-by":"publisher","DOI":"10.1515\/9781400881970-018"}],"container-title":["JCO Clinical Cancer Informatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/ascopubs.org\/doi\/pdfdirect\/10.1200\/CCI-25-00141","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,10]],"date-time":"2026-06-10T20:00:21Z","timestamp":1781121621000},"score":1,"resource":{"primary":{"URL":"https:\/\/ascopubs.org\/doi\/10.1200\/CCI-25-00141"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,4]]},"references-count":46,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2026,4]]}},"alternative-id":["10.1200\/CCI-25-00141"],"URL":"https:\/\/doi.org\/10.1200\/cci-25-00141","relation":{},"ISSN":["2473-4276"],"issn-type":[{"value":"2473-4276","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,4]]},"assertion":[{"value":"2025-05-15","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-03-05","order":1,"name":"revised","label":"Revised","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-05-05","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-06-10","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"e2500141"}}