{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,29]],"date-time":"2026-08-29T15:19:32Z","timestamp":1788016772395,"version":"build-2784847793"},"reference-count":49,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2025,8,25]],"date-time":"2025-08-25T00:00:00Z","timestamp":1756080000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Tomography"],"abstract":"<jats:p>This review explores how multimodal foundation models (MFMs) are transforming biliary tract cancer (BTC) research. BTCs are aggressive malignancies with poor prognosis, presenting unique challenges due to difficult diagnostic methods, molecular complexity, and rarity. Importantly, intrahepatic cholangiocarcinoma (iCCA), perihilar cholangiocarcinoma (pCCA), and distal bile duct cholangiocarcinoma (dCCA) represent fundamentally distinct clinical entities, with iCCA presenting as mass-forming lesions amenable to biopsy and targeted therapies, while pCCA manifests as infiltrative bile duct lesions with challenging diagnosis and primarily palliative management approaches. MFMs offer potential to advance research by integrating radiological images, histopathology, multi-omics profiles, and clinical data into unified computational frameworks, with applications tailored to these distinct BTC subtypes. Key applications include enhanced biomarker discovery that identifies previously unrecognizable cross-modal patterns, potential for improving currently limited diagnostic accuracy\u2014though validation in BTC-specific cohorts remains essential\u2014accelerated drug repurposing, and advanced patient stratification for personalized treatment. Despite promising results, challenges such as data scarcity, high computational demands, and clinical workflow integration remain to be addressed. Future research should focus on standardized data protocols, architectural innovations, and prospective validation studies. The integration of artificial intelligence (AI)-based methodologies offers new solutions for these historically challenging malignancies. However, current evidence for BTC-specific applications remains largely theoretical, with most studies limited to proof-of-concept designs or related cancer types. Comprehensive clinical validation studies and prospective trials demonstrating patient benefit are essential prerequisites for clinical implementation. The timeline for evidence-based clinical adoption likely extends 7\u201310 years, contingent on successful completion of validation studies addressing current evidence gaps.<\/jats:p>","DOI":"10.3390\/tomography11090096","type":"journal-article","created":{"date-parts":[[2025,8,26]],"date-time":"2025-08-26T06:26:31Z","timestamp":1756189591000},"page":"96","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["Leveraging Multimodal Foundation Models in Biliary Tract Cancer Research"],"prefix":"10.3390","volume":"11","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5848-7072","authenticated-orcid":false,"given":"Yashbir","family":"Singh","sequence":"first","affiliation":[{"name":"Radiology, Mayo Clinic, Rochester, MN 55905, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1760-5244","authenticated-orcid":false,"given":"Jesper B.","family":"Andersen","sequence":"additional","affiliation":[{"name":"Biotech Research and Innovation Centre (BRIC), Department of Health and Medical Sciences, University of Copenhagen, DK-1165 Copenhagen, Denmark"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8226-2319","authenticated-orcid":false,"given":"Quincy A.","family":"Hathaway","sequence":"additional","affiliation":[{"name":"Department of Radiology, University of Pennsylvania, Philadelphia, PA 19104, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0520-3582","authenticated-orcid":false,"given":"Diana V.","family":"Vera-Garcia","sequence":"additional","affiliation":[{"name":"Division of Gastroenterology & Hepatology, Mayo Clinic, Jacksonville, FL 32224, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Varekan","family":"Keishing","sequence":"additional","affiliation":[{"name":"Radiology, Mayo Clinic, Rochester, MN 55905, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7514-1030","authenticated-orcid":false,"given":"Sudhakar K.","family":"Venkatesh","sequence":"additional","affiliation":[{"name":"Radiology, Mayo Clinic, Rochester, MN 55905, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-3223-9059","authenticated-orcid":false,"given":"Sara","family":"Salehi","sequence":"additional","affiliation":[{"name":"Radiology, Mayo Clinic, Rochester, MN 55905, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Davide","family":"Povero","sequence":"additional","affiliation":[{"name":"Division of Gastroenterology & Hepatology, Mayo Clinic, Rochester, MN 55905, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6446-5785","authenticated-orcid":false,"given":"Michael B.","family":"Wallace","sequence":"additional","affiliation":[{"name":"Division of Gastroenterology & Hepatology, Mayo Clinic, Jacksonville, FL 32224, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3151-009X","authenticated-orcid":false,"given":"Gregory J.","family":"Gores","sequence":"additional","affiliation":[{"name":"Division of Gastroenterology & Hepatology, Mayo Clinic, Rochester, MN 55905, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9342-0498","authenticated-orcid":false,"given":"Yujia","family":"Wei","sequence":"additional","affiliation":[{"name":"Radiology, Mayo Clinic, Rochester, MN 55905, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Natally","family":"Horvat","sequence":"additional","affiliation":[{"name":"Radiology, Mayo Clinic, Rochester, MN 55905, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7926-6095","authenticated-orcid":false,"given":"Bradley J.","family":"Erickson","sequence":"additional","affiliation":[{"name":"Radiology, Mayo Clinic, Rochester, MN 55905, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2020-9365","authenticated-orcid":false,"given":"Emilio","family":"Quaia","sequence":"additional","affiliation":[{"name":"Department of Radiology, Universit\u00e0 di Padova, 35122 Padova, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,8,25]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"557","DOI":"10.1038\/s41575-020-0310-z","article-title":"Cholangiocarcinoma 2020: The next horizon in mechanisms and management","volume":"17","author":"Banales","year":"2020","journal-title":"Nat. Rev. Gastroenterol. Hepatol."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"428","DOI":"10.1016\/S0140-6736(21)00153-7","article-title":"Biliary tract cancer","volume":"397","author":"Valle","year":"2021","journal-title":"Lancet"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"40","DOI":"10.21037\/tgh.2018.07.02","article-title":"Precision medicine in cholangiocarcinoma","volume":"3","author":"Pellino","year":"2018","journal-title":"Transl. Gastroenterol. Hepatol."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1003","DOI":"10.1038\/ng.3375","article-title":"Genomic spectra of biliary tract cancer","volume":"47","author":"Nakamura","year":"2015","journal-title":"Nat. Genet."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Li, C., Gan, Z., Yang, Z., Yang, J., Li, L., Wang, L., and Gao, J. (2023). Multimodal Foundation Models: From Specialists to General-Purpose Assistants. arXiv.","DOI":"10.1561\/9781638283379"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"667","DOI":"10.1016\/j.jhep.2024.05.017","article-title":"A multimodal atlas of hepatocellular carcinoma reveals convergent evolutionary paths and \u2018bad apple\u2019 effect on clinical trajectory","volume":"81","author":"Chen","year":"2024","journal-title":"J. Hepatol."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"838","DOI":"10.1158\/1078-0432.CCR-22-2036","article-title":"FDA approval summary: Pemigatinib for previously treated, unresectable locally advanced or metastatic cholangiocarcinoma with FGFR2 fusion or other rearrangement","volume":"29","author":"Patel","year":"2023","journal-title":"Clin. Cancer Res."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1669","DOI":"10.1001\/jamaoncol.2021.3836","article-title":"Final overall survival efficacy results of ivosidenib for patients with advanced cholangiocarcinoma with IDH1 mutation: The phase 3 randomized clinical ClarIDHy trial","volume":"7","author":"Zhu","year":"2021","journal-title":"JAMA Oncol."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"168","DOI":"10.1016\/j.gie.2014.09.017","article-title":"Comparative effectiveness of biliary brush cytology and intraductal biopsy for detection of malignant biliary strictures: A systematic review and meta-analysis","volume":"81","author":"Navaneethan","year":"2015","journal-title":"Gastrointest. Endosc."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"100060","DOI":"10.1016\/j.esmorw.2024.100060","article-title":"Real-world treatment patterns and challenges of patients with biliary tract cancer: Retrospective chart review survey in Europe (GARNET-2)","volume":"5","author":"Bridgewater","year":"2024","journal-title":"ESMO Real World Data Digit. Oncol."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"686","DOI":"10.1038\/nature22985","article-title":"Correction: Corrigendum: Dermatologist-level classification of skin cancer with deep neural networks","volume":"546","author":"Esteva","year":"2017","journal-title":"Nature"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1148\/radiol.220152","article-title":"Pancreatic cancer detection on CT scans with deep learning: A nationwide population-based study","volume":"306","author":"Chen","year":"2023","journal-title":"Radiology"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Venugopalan, J., Tong, L., Hassanzadeh, H.R., and Wang, M.D. (2021). Multimodal deep learning models for early detection of Alzheimer\u2019s disease stage. Sci. Rep., 11.","DOI":"10.1038\/s41598-020-74399-w"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"943","DOI":"10.1016\/j.gie.2013.11.001","article-title":"Fluorescence in situ hybridization for diagnosis of cholangiocarcinoma in primary sclerosing cholangitis: A systematic review and meta-analysis","volume":"79","author":"Navaneethan","year":"2014","journal-title":"Gastrointest. Endosc."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1853","DOI":"10.1016\/S0140-6736(23)00727-4","article-title":"Pembrolizumab in combination with gemcitabine and cisplatin compared with gemcitabine and cisplatin alone for patients with advanced biliary tract cancer (KEYNOTE-966): A randomised, double-blind, placebo-controlled, phase 3 trial","volume":"401","author":"Kelley","year":"2023","journal-title":"Lancet"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"e004892","DOI":"10.1136\/jitc-2022-004892","article-title":"Multimodule characterization of immune subgroups in intrahepatic cholangiocarcinoma reveals distinct therapeutic vulnerabilities","volume":"10","author":"Lin","year":"2022","journal-title":"J. Immunother. Cancer"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1002\/hep.32125","article-title":"Early and accurate detection of cholangiocarcinoma in patients with primary sclerosing cholangitis by methylation markers in bile","volume":"75","author":"Vedeld","year":"2022","journal-title":"Hepatology"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"861","DOI":"10.1016\/j.jhep.2024.10.043","article-title":"Circulating tumor DNA status and dynamics predict recurrence in patients with resected extrahepatic cholangiocarcinoma","volume":"82","author":"Yoo","year":"2025","journal-title":"J. Hepatol."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1038\/s44287-023-00012-7","article-title":"Artificial intelligence applications in histopathology","volume":"1","author":"Bahadir","year":"2024","journal-title":"Nat. Rev. Electr. Eng."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1193","DOI":"10.1038\/s41467-021-21467-y","article-title":"An annotation-free whole-slide training approach to pathological classification of lung cancer types using deep learning","volume":"12","author":"Chen","year":"2021","journal-title":"Nat. Commun."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"757","DOI":"10.1109\/TMI.2020.3021387","article-title":"Pathomic fusion: An integrated framework for fusing histopathology and genomic features for cancer diagnosis and prognosis","volume":"41","author":"Chen","year":"2020","journal-title":"IEEE Trans. Med. Imaging."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"847","DOI":"10.1158\/1535-7163.MCT-19-0631","article-title":"Efficacy of FGFR inhibitors and combination therapies for acquired resistance in FGFR2-fusion cholangiocarcinoma","volume":"19","author":"Krook","year":"2020","journal-title":"Mol. Cancer Ther."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1054","DOI":"10.1038\/s41591-019-0462-y","article-title":"Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer","volume":"25","author":"Kather","year":"2019","journal-title":"Nat. Med."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"2168","DOI":"10.1016\/S0140-6736(13)61903-0","article-title":"Cholangiocarcinoma","volume":"383","author":"Razumilava","year":"2014","journal-title":"Lancet"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1423","DOI":"10.1111\/jgh.15329","article-title":"Regional variation and temporal trend of primary biliary cholangitis epidemiology: A systematic review and meta-analysis","volume":"36","author":"Lv","year":"2021","journal-title":"J. Gastroenterol. Hepatol."},{"key":"ref_26","unstructured":"Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2020). An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. arXiv."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"064503","DOI":"10.1117\/1.JMI.9.6.064503","article-title":"Contrastive self-supervised learning from 100 million medical images with optional supervision","volume":"9","author":"Ghesu","year":"2022","journal-title":"J. Med. Imaging"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"305","DOI":"10.1038\/s42256-020-0186-1","article-title":"Secure, privacy-preserving and federated machine learning in medical imaging","volume":"2","author":"Kaissis","year":"2020","journal-title":"Nat. Mach. Intell."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"249","DOI":"10.18632\/oncotarget.28709","article-title":"Deep learning-based uncertainty quantification for quality assurance in hepatobiliary imaging-based techniques","volume":"16","author":"Singh","year":"2025","journal-title":"Oncotarget"},{"key":"ref_30","first-page":"100335","article-title":"First-line chemotherapy in advanced biliary tract cancer ten years after the ABC-02 trial: \u201cAnd yet it moves!\u201d","volume":"27","author":"Rizzo","year":"2021","journal-title":"Cancer Treat. Res. Commun."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"463","DOI":"10.1038\/s41573-019-0024-5","article-title":"Applications of machine learning in drug discovery and development","volume":"18","author":"Vamathevan","year":"2019","journal-title":"Nat. Rev. Drug. Discov."},{"key":"ref_32","unstructured":"Ramsundar, B., Kearnes, S., Riley, P., Webster, D., Konerding, D., and Pande, V. (2015). Massively multitask networks for drug discovery. arXiv."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1424","DOI":"10.1038\/nm.4438","article-title":"Human primary liver cancer\u2013derived organoid cultures for disease modeling and drug screening","volume":"23","author":"Broutier","year":"2017","journal-title":"Nat. Med."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1016\/j.csbj.2014.11.005","article-title":"Machine learning applications in cancer prognosis and prediction","volume":"13","author":"Kourou","year":"2015","journal-title":"Comput. Struct. Biotechnol. J."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1038\/s41573-021-00301-6","article-title":"Harnessing the predictive power of preclinical models for oncology drug development","volume":"21","author":"Honkala","year":"2022","journal-title":"Nat. Rev. Drug Discov."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1016\/j.jclinepi.2022.08.016","article-title":"Handling missing data in clinical research","volume":"151","author":"Heymans","year":"2022","journal-title":"J. Clin. Epidemiol."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"2190","DOI":"10.1002\/ijc.33013","article-title":"Efficacy and safety of pembrolizumab for the treatment of advanced biliary cancer: Results from the KEYNOTE-158 and KEYNOTE-028 studies","volume":"147","author":"Oh","year":"2020","journal-title":"Int. J. Cancer"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"e2200003","DOI":"10.1200\/PO.22.00003","article-title":"Tumor mutational burden is a potential predictive biomarker for response to immune checkpoint inhibitors in patients with advanced biliary tract cancer","volume":"6","author":"Liddell","year":"2022","journal-title":"JCO Precis. Oncol."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"e253","DOI":"10.1016\/S1470-2045(19)30154-8","article-title":"Digital pathology and artificial intelligence","volume":"20","author":"Niazi","year":"2019","journal-title":"Lancet Oncol."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"816","DOI":"10.1373\/clinchem.2010.157131","article-title":"Laboratory medicine best practices: Systematic evidence review and evaluation methods for quality improvement","volume":"57","author":"Christenson","year":"2011","journal-title":"Clin. Chem."},{"key":"ref_41","unstructured":"(2025, April 14). Deep Learning Methods for Leveraging Data Scarcity and Enforcing Prior in Healthcare Image Analysis. Available online: https:\/\/iris.unige.it\/handle\/11567\/1244197."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1038\/s41591-018-0307-0","article-title":"The practical implementation of artificial intelligence technologies in medicine","volume":"25","author":"He","year":"2019","journal-title":"Nat. Med."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"1544","DOI":"10.1001\/jamainternmed.2018.3763","article-title":"Potential biases in machine learning algorithms using electronic health record data","volume":"178","author":"Gianfrancesco","year":"2018","journal-title":"JAMA Intern. Med."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Wilhelm, C., Steckelberg, A., and Rebitschek, F.G. (2024). Is artificial intelligence for medical professionals serving the patients? Protocol for a mixed method systematic review on patient-relevant benefits and harms of algorithmic decision-making. medRxiv.","DOI":"10.1101\/2024.03.27.24304965"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"100092","DOI":"10.1016\/j.esmogo.2024.100092","article-title":"New perspectives in biliary tract cancers","volume":"5","author":"Goetze","year":"2024","journal-title":"ESMO Gastrointest. Oncol."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Azizi, S., Mustafa, B., Ryan, F., Beaver, Z., Freyberg, J., Deaton, J., Loh, A., Karthikesalingam, A., Kornblith, S., and Chen, T. (2021, January 10\u201317). Big self-supervised models advance medical image classification. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Montreal, QC, Canada.","DOI":"10.1109\/ICCV48922.2021.00346"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"e537","DOI":"10.1016\/S2589-7500(20)30218-1","article-title":"Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: The CONSORT-AI Extension","volume":"2","author":"Liu","year":"2020","journal-title":"Lancet Digit. Health"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"827","DOI":"10.1038\/s41551-020-0578-x","article-title":"Integrating spatial gene expression and breast tumour morphology via deep learning","volume":"4","author":"He","year":"2020","journal-title":"Nat. Biomed. Eng."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Xu, Q., Xie, W., Liao, B., Hu, C., Qin, L., Yang, Z., and Luo, A. (2023). Interpretability of clinical decision support systems based on artificial intelligence from technological and medical perspective: A systematic review. J. Healthc. Eng., 9919269.","DOI":"10.1155\/2023\/9919269"}],"container-title":["Tomography"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2379-139X\/11\/9\/96\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T18:32:42Z","timestamp":1760034762000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2379-139X\/11\/9\/96"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8,25]]},"references-count":49,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2025,9]]}},"alternative-id":["tomography11090096"],"URL":"https:\/\/doi.org\/10.3390\/tomography11090096","relation":{},"ISSN":["2379-139X"],"issn-type":[{"value":"2379-139X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,8,25]]}}}