{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T06:24:16Z","timestamp":1787293456604,"version":"build-2736575974"},"reference-count":52,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T00:00:00Z","timestamp":1787270400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Cell Dev. Biol."],"abstract":"<jats:sec>\n                    <jats:title>Objectives<\/jats:title>\n                    <jats:p>To retrospectively develop and evaluate a task-conditioned multimodal artificial intelligence framework, in which a learned task embedding and modality-availability mask generate endpoint-specific weights over predefined radiology-only, pathology-only, and fusion experts, for breast cancer staging and biomarker profiling.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>We retrospectively assembled a cohort of 923 patients with paired radiology and pathology data. Nine tasks included TNM components [tumor extent (T), nodal involvement (N), and distant metastasis (M)], clinical stage, histological grade, and estrogen receptor, progesterone receptor, HER2, and Ki-67 status. Five convolutional neural network backbones were benchmarked for each modality. Task-specific models were selected by validation AUROC, and a late-fusion transformer was used for paired data. We further evaluated learned MoE gating, task-conditioned MoE routing with task embeddings and modality-availability masks, missing-modality robustness, and paired-bootstrap AUROC comparisons. An independent external cohort assessed feasibility for mammography-based pathological complete response and pathology-based triple-negative breast cancer (TNBC) prediction.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Best single-modality models achieved AUROCs ranging from 0.606 to 0.990 across tasks, with an AUROC of 0.990 observed for M staging and strong performance observed for histological grade (AUROC, 0.949; accuracy, 0.908) and HER2 status (AUROC, 0.810; accuracy, 0.762). Fixed late fusion improved the mean AUROC from 0.813 to 0.826. Learned MoE gating and task-conditioned MoE further improved mean AUROC to 0.830 and 0.834, respectively. Task-conditioned MoE improved mean AUROC by 0.009 over fixed late fusion (95% CI, 0.005\u20130.013; P = 0.011) and by 0.021 over the best single-modality expert. Under 70% random missing-modality simulation, task-conditioned MoE outperformed fixed fusion with zero imputation by 0.067 mean AUROC. In external feasibility analyses, the mammography-based pCR model achieved an AUROC of 0.758, and the pathology-based triple-negative breast cancer model achieved an AUROC of 0.872.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>OmniBreast demonstrated the feasibility of task-conditioned radiology-only, pathology-only, and fused prediction in the internal nine-task benchmark. The external analyses provided preliminary support for feasibility on two related endpoint-extension tasks but did not establish generalizability across the nine primary tasks; multicenter external validation and prospective clinical evaluation remain necessary.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/fcell.2026.1863777","type":"journal-article","created":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T05:37:14Z","timestamp":1787290634000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Development and internal validation of a task-conditioned multimodal radiology\u2013pathology model for breast cancer staging and biomarker profiling: a retrospective cohort study"],"prefix":"10.3389","volume":"14","author":[{"given":"Yang","family":"Liu","sequence":"first","affiliation":[{"name":"Department of Medical Oncology, The First Affiliated Hospital of Bengbu Medical University","place":["Bengbu, Anhui, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yunfei","family":"Dong","sequence":"additional","affiliation":[{"name":"Faculty of Computing, Harbin Institute of Technology","place":["Harbin, Heilongjiang, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Guiyun","family":"Zhang","sequence":"additional","affiliation":[{"name":"Department of Medical Oncology, The First Affiliated Hospital of Bengbu Medical University","place":["Bengbu, Anhui, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yue","family":"Wu","sequence":"additional","affiliation":[{"name":"Department of Medical Oncology, The First Affiliated Hospital of Bengbu Medical University","place":["Bengbu, Anhui, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shanshan","family":"Zhang","sequence":"additional","affiliation":[{"name":"Department of Medical Oncology, The First Affiliated Hospital of Bengbu Medical University","place":["Bengbu, Anhui, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yufo","family":"Chen","sequence":"additional","affiliation":[{"name":"Department of Medical Oncology, The First Affiliated Hospital of Bengbu Medical University","place":["Bengbu, Anhui, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ruirui","family":"Cao","sequence":"additional","affiliation":[{"name":"Department of Medical Oncology, The First Affiliated Hospital of Bengbu Medical University","place":["Bengbu, Anhui, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rui","family":"Wang","sequence":"additional","affiliation":[{"name":"Department of Medical Oncology, The First Affiliated Hospital of Bengbu Medical University","place":["Bengbu, Anhui, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chengliang","family":"Yin","sequence":"additional","affiliation":[{"name":"Department of Biomedical Engineering, Faculty of Engineering, Universiti Malaya","place":["Kuala Lumpur, Malaysia"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yumei","family":"Li","sequence":"additional","affiliation":[{"name":"Department of Medical Oncology, The First Affiliated Hospital of Bengbu Medical University","place":["Bengbu, Anhui, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2026,8,21]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"131","DOI":"10.1097\/PAI.0000000000001258","article-title":"Toward accurate deep learning-based prediction of Ki67, ER, PR, and HER2 status from H&E-Stained breast cancer images","volume":"33","author":"Akbarnejad","year":"2025","journal-title":"Appl. Immunohistochem. and Mol. Morphol."},{"key":"B2","doi-asserted-by":"publisher","first-page":"e45547","DOI":"10.2196\/45547","article-title":"Data-efficient computational pathology platform for faster and cheaper breast cancer subtype identifications: development of a deep learning model","volume":"9","author":"Bae","year":"2023","journal-title":"JMIR Cancer"},{"key":"B3","first-page":"15016","article-title":"Learning to exploit temporal structure for biomedical vision-language processing","volume-title":"Inproceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Bannur","year":"2023"},{"key":"B4","article-title":"Monai: an open-source framework for deep learning in healthcare","author":"Cardoso","year":"2022","journal-title":"arXiv Preprint arXiv:2211.02701"},{"key":"B5","doi-asserted-by":"publisher","first-page":"116","DOI":"10.1186\/s41747-025-00653-3","article-title":"Deep-learning prediction of breast cancer hormone receptor status from CEM: a preliminary study","volume":"9","author":"Carriero","year":"2025","journal-title":"Eur. Radiology Experimental"},{"key":"B6","doi-asserted-by":"publisher","first-page":"865","DOI":"10.1016\/j.ccell.2022.07.004","article-title":"Pan-cancer integrative histology-genomic analysis via multimodal deep learning","volume":"40","author":"Chen","year":"2022","journal-title":"Cancer Cell"},{"key":"B7","doi-asserted-by":"publisher","first-page":"23","DOI":"10.1016\/j.breastdis.2016.01.040","article-title":"Radiomics and radiogenomics in breast cancer","volume":"27","author":"Colen","year":"2016","journal-title":"Breast Dis."},{"key":"B8","doi-asserted-by":"publisher","first-page":"30","DOI":"10.1038\/s41523-018-0079-1","article-title":"Image analysis with deep learning to predict breast cancer grade, ER status, histologic subtype, and intrinsic subtype","volume":"4","author":"Couture","year":"2018","journal-title":"NPJ Breast Cancer"},{"key":"B9","article-title":"Multi-task learning with deep neural networks: a survey","author":"Crawshaw","year":"2020"},{"key":"B10","first-page":"702","article-title":"Randaugment: practical automated data augmentation with a reduced search space","volume-title":"Inproceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops","author":"Cubuk","year":"2020"},{"key":"B11","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41591-025-03982-3","article-title":"A multimodal whole-slide foundation model for pathology","volume":"5","author":"Ding","year":"2025","journal-title":"Nat. Medicine"},{"key":"B12","doi-asserted-by":"publisher","first-page":"2199","DOI":"10.1001\/jama.2017.14585","article-title":"Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer","volume":"318","author":"Ehteshami Bejnordi","year":"2017","journal-title":"Jama"},{"key":"B13","doi-asserted-by":"publisher","first-page":"917","DOI":"10.1038\/s41591-024-03408-6","article-title":"Nationwide real-world implementation of AI for cancer detection in population-based mammography screening","volume":"31","author":"Eisemann","year":"2025","journal-title":"Nat. Medicine"},{"key":"B14","first-page":"27503","article-title":"Efficiently identifying task groupings for multi-task learning","volume":"34","author":"Fifty","year":"2021","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"B15","doi-asserted-by":"publisher","first-page":"e2437402","DOI":"10.1001\/jamanetworkopen.2024.37402","article-title":"Artificial intelligence algorithm for subclinical breast cancer detection","volume":"7","author":"Gjesvik","year":"2024","journal-title":"JAMA Netw. Open"},{"key":"B16","doi-asserted-by":"publisher","first-page":"1593143","DOI":"10.3389\/fmed.2025.1593143","article-title":"Predicting estrogen receptor status from HE-stained breast cancer slides using artificial intelligence","volume":"12","author":"H\u00f8ib\u00f8","year":"2025","journal-title":"Front. Medicine"},{"key":"B17","doi-asserted-by":"publisher","first-page":"203","DOI":"10.1038\/s41592-020-01008-z","article-title":"nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation","volume":"18","author":"Isensee","year":"2021","journal-title":"Nat. Methods"},{"key":"B18","doi-asserted-by":"publisher","first-page":"347","DOI":"10.1158\/1940-6207.capr-24-0427","article-title":"Deriving a mammogram-based risk score from screening digital breast tomosynthesis for 5-year breast cancer risk prediction","volume":"18","author":"Jiang","year":"2025","journal-title":"Cancer Prev. Res."},{"key":"B19","doi-asserted-by":"publisher","first-page":"1054","DOI":"10.1038\/s41591-019-0462-y","article-title":"Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer","volume":"25","author":"Kather","year":"2019","journal-title":"Nat. Medicine"},{"key":"B20","doi-asserted-by":"publisher","first-page":"195","DOI":"10.1186\/s12916-019-1426-2","article-title":"Key challenges for delivering clinical impact with artificial intelligence","volume":"17","author":"Kelly","year":"2019","journal-title":"BMC Medicine"},{"key":"B21","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1038\/s41698-024-00772-x","article-title":"Prediction of post-treatment recurrence in early-stage breast cancer using deep-learning with mid-infrared chemical histopathological imaging","volume":"9","author":"Keogan","year":"2025","journal-title":"NPJ Precision Oncology"},{"key":"B22","doi-asserted-by":"publisher","first-page":"171","DOI":"10.1038\/s41746-022-00712-8","article-title":"Multimodal machine learning in precision health: a scoping review","volume":"5","author":"Kline","year":"2022","journal-title":"NPJ Digital Medicine"},{"key":"B23","doi-asserted-by":"publisher","first-page":"1095","DOI":"10.1016\/j.ccell.2022.09.012","article-title":"Artificial intelligence for multimodal data integration in oncology","volume":"40","author":"Lipkova","year":"2022","journal-title":"Cancer Cell"},{"key":"B24","doi-asserted-by":"publisher","first-page":"1364","DOI":"10.1038\/s41591-020-1034-x","article-title":"SPIRIT-AI and CONSORT-AI working group. Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension","volume":"26","author":"Liu","year":"2020","journal-title":"Nat. Med."},{"key":"B25","first-page":"10012","article-title":"Swin transformer: hierarchical vision transformer using shifted windows","volume-title":"Inproceedings of the IEEE\/CVF International Conference on Computer Vision","author":"Liu","year":"2021"},{"key":"B26","doi-asserted-by":"publisher","first-page":"62","DOI":"10.1016\/j.irbm.2020.12.002","article-title":"A hybrid deep learning model for predicting molecular subtypes of human breast cancer using multimodal data","volume":"43","author":"Liu","year":"2022","journal-title":"Irbm"},{"key":"B27","first-page":"11976","article-title":"A convnet for the 2020s","volume-title":"Inproceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Liu","year":"2022"},{"key":"B28","doi-asserted-by":"publisher","first-page":"863","DOI":"10.1038\/s41591-024-02856-4","article-title":"A visual-language foundation model for computational pathology","volume":"30","author":"Lu","year":"2024","journal-title":"Nat. Medicine"},{"key":"B29","doi-asserted-by":"publisher","first-page":"259","DOI":"10.1038\/s41698-025-01057-7","article-title":"Multimodal prediction based on ultrasound for response to neoadjuvant chemotherapy in triple negative breast cancer","volume":"9","author":"Lyu","year":"2025","journal-title":"Npj Precis. Oncol."},{"key":"B30","doi-asserted-by":"publisher","first-page":"1496215","DOI":"10.3389\/fonc.2025.1496215","article-title":"Multimodal BEHRT: transformers for multimodal electronic health records to predict breast cancer prognosis","volume":"15","author":"Mbaye","year":"2025","journal-title":"Front. Oncol."},{"key":"B31","doi-asserted-by":"publisher","first-page":"89","DOI":"10.1038\/s41586-019-1799-6","article-title":"International evaluation of an AI system for breast cancer screening","volume":"577","author":"McKinney","year":"2020","journal-title":"Nature"},{"key":"B32","doi-asserted-by":"publisher","first-page":"774","DOI":"10.1186\/s12967-025-06617-w","article-title":"Attention-based multimodal deep learning for interpretable and generalizable prediction of pathological complete response in breast cancer","volume":"23","author":"Nishizawa","year":"2025","journal-title":"J. Transl. Med."},{"key":"B33","doi-asserted-by":"publisher","first-page":"319","DOI":"10.1016\/j.jacr.2023.10.018","article-title":"Artificial intelligence-driven mammography-based future breast cancer risk prediction: a systematic review","volume":"21","author":"Schopf","year":"2024","journal-title":"J. Am. Coll. Radiology"},{"key":"B34","doi-asserted-by":"publisher","first-page":"121526","DOI":"10.1016\/j.eswa.2023.121526","article-title":"Miter: medical image\u2013text joint adaptive pretraining with multi-level contrastive learning","volume":"238","author":"Shu","year":"2024","journal-title":"Expert Syst. Appl."},{"key":"B35","doi-asserted-by":"publisher","first-page":"100847","DOI":"10.1016\/j.modpat.2025.100847","article-title":"Computational pathology for accurate prediction of breast cancer recurrence: development and validation of a deep learning-based tool","volume":"16","author":"Su","year":"2025","journal-title":"Mod. Pathol."},{"key":"B36","doi-asserted-by":"publisher","first-page":"209","DOI":"10.3322\/caac.21660","article-title":"Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries","volume":"71","author":"Sung","year":"2021","journal-title":"CA A Cancer Journal Clinicians"},{"key":"B37","first-page":"6105","article-title":"Efficientnet: rethinking model scaling for convolutional neural networks","volume-title":"Ininternational Conference on Machine Learning","author":"Tan","year":"2019"},{"key":"B38","doi-asserted-by":"publisher","first-page":"599","DOI":"10.1038\/s41591-024-03302-1","article-title":"Collaboration between Clinicians and vision\u2013language models in radiology report generation","volume":"31","author":"Tanno","year":"2025","journal-title":"Nat. Med."},{"key":"B39","doi-asserted-by":"publisher","first-page":"135","DOI":"10.1159\/000516114","article-title":"St. gallen\/vienna 2021: a brief summary of the consensus discussion on customizing therapies for women with early breast cancer","volume":"16","author":"Thomssen","year":"2021","journal-title":"Breast Care"},{"key":"B40","doi-asserted-by":"publisher","first-page":"1399","DOI":"10.1038\/s41551-022-00936-9","article-title":"Expert-level detection of pathologies from unannotated chest X-ray images via self-supervised learning","volume":"6","author":"Tiu","year":"2022","journal-title":"Nat. Biomedical Engineering"},{"key":"B41","doi-asserted-by":"publisher","first-page":"44","DOI":"10.1038\/s41591-018-0300-7","article-title":"High-performance medicine: the convergence of human and artificial intelligence","volume":"25","author":"Topol","year":"2019","journal-title":"Nat. Medicine"},{"key":"B42","doi-asserted-by":"publisher","first-page":"48","DOI":"10.1038\/s41746-022-00592-y","article-title":"Machine learning for medical imaging: methodological failures and recommendations for the future","volume":"5","author":"Varoquaux","year":"2022","journal-title":"NPJ Digital Medicine"},{"key":"B43","doi-asserted-by":"publisher","first-page":"2924","DOI":"10.1038\/s41591-024-03141-0","article-title":"A foundation model for clinical-grade computational pathology and rare cancers detection","volume":"30","author":"Vorontsov","year":"2024","journal-title":"Nat. Medicine"},{"key":"B44","first-page":"3876","article-title":"Medclip: contrastive learning from unpaired medical images and text","volume-title":"Inproceedings of the 2022 Conference on Empirical Methods in Natural Language Processing","author":"Wang","year":"2022"},{"key":"B45","doi-asserted-by":"publisher","first-page":"1364","DOI":"10.5858\/arpa.2018-0902-sa","article-title":"Human epidermal growth factor receptor 2 testing in breast cancer: American society of clinical oncology\/college of American pathologists clinical practice guideline focused update","volume":"142","author":"Wolff","year":"2018","journal-title":"Archives Pathology and Laboratory Medicine"},{"key":"B46","first-page":"21372","article-title":"Medklip: medical knowledge enhanced language-image pre-training for x-ray diagnosis","volume-title":"Inproceedings of the IEEE\/CVF International Conference on Computer Vision","author":"Wu","year":"2023"},{"key":"B47","doi-asserted-by":"publisher","first-page":"181","DOI":"10.1038\/s41586-024-07441-w","article-title":"A whole-slide foundation model for digital pathology from real-world data","volume":"630","author":"Xu","year":"2024","journal-title":"Nature"},{"key":"B48","doi-asserted-by":"publisher","first-page":"e1002683","DOI":"10.1371\/journal.pmed.1002683","article-title":"Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: a cross-sectional study","volume":"15","author":"Zech","year":"2018","journal-title":"PLoS Medicine"},{"key":"B49","doi-asserted-by":"publisher","first-page":"16","DOI":"10.1038\/s41523-023-00517-2","article-title":"Predicting breast cancer types on and beyond molecular level in a multi-modal fashion","volume":"9","author":"Zhang","year":"2023","journal-title":"NPJ Breast Cancer"},{"key":"B50","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1186\/s13058-025-02181-9","article-title":"Attention-based multimodal fusion transformer for predicting the efficacy of neoadjuvant therapy in breast cancer: a cross-institutional retrospective study","volume":"28","author":"Zhang","year":"2025","journal-title":"Breast Cancer Res."},{"key":"B51","doi-asserted-by":"publisher","first-page":"432","DOI":"10.1038\/s41746-025-01807-8","article-title":"Vision-language model for report generation and outcome prediction in CT pulmonary angiogram","volume":"8","author":"Zhong","year":"2025","journal-title":"NPJ Digital Medicine"},{"key":"B52","doi-asserted-by":"publisher","first-page":"550","DOI":"10.1148\/radiol.2021203758","article-title":"Deep learning predicts interval and screening-detected cancer from screening mammograms: a case-case-control study in 6369 women","volume":"301","author":"Zhu","year":"2021","journal-title":"Radiology"}],"container-title":["Frontiers in Cell and Developmental Biology"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fcell.2026.1863777\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T05:37:17Z","timestamp":1787290637000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fcell.2026.1863777\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,8,21]]},"references-count":52,"alternative-id":["10.3389\/fcell.2026.1863777"],"URL":"https:\/\/doi.org\/10.3389\/fcell.2026.1863777","relation":{},"ISSN":["2296-634X"],"issn-type":[{"value":"2296-634X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,8,21]]},"article-number":"1863777"}}