{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,24]],"date-time":"2026-06-24T16:50:25Z","timestamp":1782319825965,"version":"3.54.5"},"reference-count":48,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2024,12,30]],"date-time":"2024-12-30T00:00:00Z","timestamp":1735516800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0"},{"start":{"date-parts":[[2024,12,30]],"date-time":"2024-12-30T00:00:00Z","timestamp":1735516800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["npj Digit. Med."],"DOI":"10.1038\/s41746-024-01411-2","type":"journal-article","created":{"date-parts":[[2024,12,30]],"date-time":"2024-12-30T10:36:08Z","timestamp":1735554968000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":14,"title":["Aligning knowledge concepts to whole slide images for precise histopathology image analysis"],"prefix":"10.1038","volume":"7","author":[{"given":"Weiqin","family":"Zhao","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ziyu","family":"Guo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yinshuang","family":"Fan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6184-3931","authenticated-orcid":false,"given":"Yuming","family":"Jiang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9942-0109","authenticated-orcid":false,"given":"Maximus C. F.","family":"Yeung","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9315-6527","authenticated-orcid":false,"given":"Lequan","family":"Yu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2024,12,30]]},"reference":[{"key":"1411_CR1","doi-asserted-by":"publisher","first-page":"3950","DOI":"10.1109\/TCYB.2019.2935141","volume":"50","author":"X Wang","year":"2019","unstructured":"Wang, X. et al. Weakly supervised deep learning for whole slide lung cancer image analysis. IEEE Trans. Cybern. 50, 3950\u20133962 (2019).","journal-title":"IEEE Trans. Cybern."},{"key":"1411_CR2","doi-asserted-by":"publisher","first-page":"1686","DOI":"10.1016\/j.ajpath.2019.05.007","volume":"189","author":"S Wang","year":"2019","unstructured":"Wang, S., Yang, D. M., Rong, R., Zhan, X. & Xiao, G. Pathology image analysis using segmentation deep learning algorithms. Am. J. Pathol. 189, 1686\u20131698 (2019).","journal-title":"Am. J. Pathol."},{"key":"1411_CR3","doi-asserted-by":"publisher","first-page":"1559","DOI":"10.1038\/s41591-018-0177-5","volume":"24","author":"N Coudray","year":"2018","unstructured":"Coudray, N. et al. Classification and mutation prediction from non\u2013small cell lung cancer histopathology images using deep learning. Nat. Med. 24, 1559\u20131567 (2018).","journal-title":"Nat. Med."},{"key":"1411_CR4","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 770\u2013778 (IEEE, 2016).","DOI":"10.1109\/CVPR.2016.90"},{"key":"1411_CR5","doi-asserted-by":"publisher","first-page":"102032","DOI":"10.1016\/j.media.2021.102032","volume":"70","author":"A Riasatian","year":"2021","unstructured":"Riasatian, A. et al. Fine-tuning and training of DenseNet for histopathology image representation using TCGA diagnostic slides. Med. Image Anal. 70, 102032 (2021).","journal-title":"Med. Image Anal."},{"key":"1411_CR6","doi-asserted-by":"crossref","unstructured":"Chen, R. J. et al. Scaling vision transformers to gigapixel images via hierarchical self-supervised learning. In 2022 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 16144\u201316155 (IEEE, 2022).","DOI":"10.1109\/CVPR52688.2022.01567"},{"key":"1411_CR7","doi-asserted-by":"publisher","first-page":"102559","DOI":"10.1016\/j.media.2022.102559","volume":"81","author":"X Wang","year":"2022","unstructured":"Wang, X. et al. Transformer-based unsupervised contrastive learning for histopathological image classification. Med. Image Anal. 81, 102559 (2022).","journal-title":"Med. Image Anal."},{"key":"1411_CR8","unstructured":"Ilse, M., Tomczak, J. & Welling, M. Attention-based deep multiple instance learning. In Proc. 35th International Conference on Machine Learning 2127\u20132136 (PMLR, 2018)."},{"key":"1411_CR9","doi-asserted-by":"crossref","unstructured":"Hou, W. et al. H2-mil: Exploring hierarchical representation with heterogeneous multiple instance learning for whole slide image analysis. In Proc. AAAI Conference on Artificial Intelligence 933\u2013941 (AAAI Press, 2022).","DOI":"10.1609\/aaai.v36i1.19976"},{"key":"1411_CR10","doi-asserted-by":"crossref","unstructured":"Guan, Y. et al. Node-aligned graph convolutional network for whole-slide image representation and classification. In 2022 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 18813\u201318823 (IEEE, 2022).","DOI":"10.1109\/CVPR52688.2022.01825"},{"key":"1411_CR11","doi-asserted-by":"crossref","unstructured":"Chan, T. H., Cendra, F. J., Ma, L., Yin, G. & Yu, L. Histopathology whole slide image analysis with heterogeneous graph representation learning. In 2023 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 15661\u201315670 (IEEE, 2023).","DOI":"10.1109\/CVPR52729.2023.01503"},{"key":"1411_CR12","unstructured":"Qu, L., Luo, X., Fu, K., Wang, M. & Song, Z. The rise of ai language pathologists: exploring two-level prompt learning for few-shot weakly-supervised whole slide image classification. In 2023 Advances in Neural Information Processing Systems 36 (Curran Associates, 2023)."},{"key":"1411_CR13","doi-asserted-by":"crossref","unstructured":"Gamper, J. & Rajpoot, N. Multiple instance captioning: Learning representations from histopathology textbooks and articles. 16549\u201316559 (2021).","DOI":"10.1109\/CVPR46437.2021.01628"},{"key":"1411_CR14","doi-asserted-by":"crossref","unstructured":"Lu, M. Y. et al. Visual language pretrained multiple instance zero-shot transfer for histopathology images. In 2023 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 19764\u201319775 (IEEE, 2023).","DOI":"10.1109\/CVPR52729.2023.01893"},{"key":"1411_CR15","unstructured":"Radford, A. et al. Learning transferable visual models from natural language supervision. In Proc. 38th International Conference on Machine Learning 8748\u20138763 (PMLR, 2021)."},{"key":"1411_CR16","doi-asserted-by":"crossref","unstructured":"Huang, Z., Bianchi, F., Yuksekgonul, M., Montine, T. & Zou, J. A visual-language foundation model for pathology image analysis using medical twitter. Nat. Med. 29, 2307\u20132316 (2023).","DOI":"10.1038\/s41591-023-02504-3"},{"key":"1411_CR17","unstructured":"Ikezogwo, W. O. et al. Quilt-1m: one million image-text pairs for histopathology. In 2024 Advances in Neural Information Processing Systems 36 (Curran Associates, 2024)."},{"key":"1411_CR18","doi-asserted-by":"crossref","unstructured":"Lu, M. Y. et al. A visual-language foundation model for computational pathology. Nat. Med. 30, 863\u2013874 (2024).","DOI":"10.1038\/s41591-024-02856-4"},{"key":"1411_CR19","doi-asserted-by":"crossref","unstructured":"Truhn, D., Reis-Filho, J. S. & Kather, J. N. Large language models should be used as scientific reasoning engines, not knowledge databases. Nat. Med. 29, 2983\u20132984 (2023).","DOI":"10.1038\/s41591-023-02594-z"},{"key":"1411_CR20","doi-asserted-by":"publisher","first-page":"773","DOI":"10.1038\/d41586-023-00816-5","volume":"615","author":"K Sanderson","year":"2023","unstructured":"Sanderson, K. Gpt-4 is here: what scientists think. Nature 615, 773 (2023).","journal-title":"Nature"},{"key":"1411_CR21","doi-asserted-by":"publisher","first-page":"686","DOI":"10.1038\/s41416-020-01122-x","volume":"124","author":"A Echle","year":"2021","unstructured":"Echle, A. et al. Deep learning in cancer pathology: a new generation of clinical biomarkers. Br. J. Cancer 124, 686\u2013696 (2021).","journal-title":"Br. J. Cancer"},{"key":"1411_CR22","doi-asserted-by":"publisher","DOI":"10.1038\/s41523-023-00518-1","volume":"9","author":"D Mandair","year":"2023","unstructured":"Mandair, D., Reis-Filho, J. S. & Ashworth, A. Biological insights and novel biomarker discovery through deep learning approaches in breast cancer histopathology. NPJ Breast Cancer 9, 21 (2023).","journal-title":"NPJ Breast Cancer"},{"key":"1411_CR23","doi-asserted-by":"publisher","first-page":"3997","DOI":"10.1200\/JCO.2013.50.9984","volume":"31","author":"A Wolff","year":"2013","unstructured":"Wolff, A. et al. American society of clinical oncology; college of american pathologists. recommendations for human epidermal growth factor receptor 2 testing in breast cancer: American Society of Clinical Oncology\/College of American Pathologists clinical practice guideline update. J. Clin. Oncol. 31, 3997\u20134013 (2013).","journal-title":"J. Clin. Oncol."},{"key":"1411_CR24","doi-asserted-by":"publisher","first-page":"102486","DOI":"10.1016\/j.media.2022.102486","volume":"80","author":"W Lu","year":"2022","unstructured":"Lu, W. et al. Slidegraph+: whole slide image level graphs to predict her2 status in breast cancer. Med. Image Anal. 80, 102486 (2022).","journal-title":"Med. Image Anal."},{"key":"1411_CR25","doi-asserted-by":"publisher","first-page":"292","DOI":"10.1200\/EDBK_175231","volume":"37","author":"RJ Kelly","year":"2017","unstructured":"Kelly, R. J. Immunotherapy for esophageal and gastric cancer. Am. Soc. Clin. Oncol. Educ. book 37, 292\u2013300 (2017).","journal-title":"Am. Soc. Clin. Oncol. Educ. book"},{"key":"1411_CR26","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-021-02168-4","volume":"11","author":"M Hinata","year":"2021","unstructured":"Hinata, M. & Ushiku, T. Detecting immunotherapy-sensitive subtype in gastric cancer using histologic image-based deep learning. Sci. Rep. 11, 22636 (2021).","journal-title":"Sci. Rep."},{"key":"1411_CR27","doi-asserted-by":"publisher","first-page":"353","DOI":"10.1007\/s00428-019-02724-4","volume":"476","author":"M Fukayama","year":"2020","unstructured":"Fukayama, M. et al. Thirty years of Epstein-Barr virus-associated gastric carcinoma. Virchows Arch. 476, 353\u2013365 (2020).","journal-title":"Virchows Arch."},{"key":"1411_CR28","doi-asserted-by":"publisher","first-page":"641","DOI":"10.1097\/01.MP.0000076980.73826.C0","volume":"16","author":"KL Grogg","year":"2003","unstructured":"Grogg, K. L., Lohse, C. M., Pankratz, V. S., Halling, K. C. & Smyrk, T. C. Lymphocyte-rich gastric cancer: associations with Epstein-Barr virus, microsatellite instability, histology, and survival. Mod. Pathol. 16, 641\u2013651 (2003).","journal-title":"Mod. Pathol."},{"key":"1411_CR29","doi-asserted-by":"publisher","first-page":"505","DOI":"10.1007\/s10120-012-0226-6","volume":"16","author":"T Arai","year":"2013","unstructured":"Arai, T. et al. Frequent microsatellite instability in papillary and solid-type, poorly differentiated adenocarcinomas of the stomach. Gastric Cancer 16, 505\u2013512 (2013).","journal-title":"Gastric Cancer"},{"key":"1411_CR30","doi-asserted-by":"publisher","first-page":"101789","DOI":"10.1016\/j.media.2020.101789","volume":"65","author":"J Yao","year":"2020","unstructured":"Yao, J., Zhu, X., Jonnagaddala, J., Hawkins, N. J. & Huang, J. Whole slide images based cancer survival prediction using attention guided deep multiple instance learning networks. Med. Image Anal. 65, 101789 (2020).","journal-title":"Med. Image Anal."},{"key":"1411_CR31","doi-asserted-by":"publisher","first-page":"555","DOI":"10.1038\/s41551-020-00682-w","volume":"5","author":"MY Lu","year":"2021","unstructured":"Lu, M. Y. et al. Data-efficient and weakly supervised computational pathology on whole-slide images. Nat. Biomed. Eng. 5, 555\u2013570 (2021).","journal-title":"Nat. Biomed. Eng."},{"key":"1411_CR32","doi-asserted-by":"crossref","unstructured":"Zheng, Y., Gindra, R., Betke, M., Beane, J. E. & Kolachalama, V. B. A deep learning based graph-transformer for whole slide image classification. IEEE Transactions on Medical Imaging 41, 3003\u20133015 (IEEE, 2022).","DOI":"10.1109\/TMI.2022.3176598"},{"key":"1411_CR33","first-page":"2136","volume":"34","author":"Z Shao","year":"2021","unstructured":"Shao, Z. et al. Transmil: Transformer based correlated multiple instance learning for whole slide image classification. Adv. Neural Inf. Process. Syst. 34, 2136\u20132147 (2021).","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"1411_CR34","doi-asserted-by":"crossref","unstructured":"Yang, Y. et al. Language in a bottle: Language model guided concept bottlenecks for interpretable image classification. In 2023 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 19187\u201319197 (IEEE, 2023).","DOI":"10.1109\/CVPR52729.2023.01839"},{"key":"1411_CR35","doi-asserted-by":"crossref","unstructured":"Yan, A. et al. Learning concise and descriptive attributes for visual recognition. In 2023 IEEE\/CVF International Conference on Computer Vision (ICCV) 3090\u20133100 (IEEE, 2023).","DOI":"10.1109\/ICCV51070.2023.00287"},{"key":"1411_CR36","doi-asserted-by":"publisher","unstructured":"Ratti, M., Lampis, A. & Hahne, J. C. Microsatellite instability in gastric cancer: molecular bases, clinical perspectives, and new treatment approaches. Cell. Mol. Life Sci. https:\/\/doi.org\/10.1007\/s00018-018-2906-9 (2018).","DOI":"10.1007\/s00018-018-2906-9"},{"key":"1411_CR37","doi-asserted-by":"crossref","unstructured":"Zheng, S. et al. Benchmarking pathclip for pathology image analysis. Journal of Imaging Informatics in Medicine 4, 1\u201317, (Springer, 2024).","DOI":"10.1007\/s10278-024-01128-4"},{"key":"1411_CR38","doi-asserted-by":"crossref","unstructured":"Lu, M. Y. et al. A multimodal generative ai copilot for human pathology. Nature 634, 466\u2013473 (2024).","DOI":"10.1038\/s41586-024-07618-3"},{"key":"1411_CR39","doi-asserted-by":"publisher","first-page":"2337","DOI":"10.1007\/s11263-022-01653-1","volume":"130","author":"K Zhou","year":"2022","unstructured":"Zhou, K., Yang, J., Loy, C. C. & Liu, Z. Learning to prompt for vision-language models. Int. J. Comput. Vis. 130, 2337\u20132348 (2022).","journal-title":"Int. J. Comput. Vis."},{"key":"1411_CR40","doi-asserted-by":"crossref","unstructured":"Gao, P. et al. Clip-adapter: better vision-language models with feature adapters. International Journal of Computer Vision 132, 581\u2013595 (Springer, 2024).","DOI":"10.1007\/s11263-023-01891-x"},{"key":"1411_CR41","unstructured":"Maron, O. & Lozano-P\u00e9rez, T. Attention is all you need. In 2017 Advances in Neural Information Processing Systems 30570\u2013576 (Curran Associates, 2017)."},{"key":"1411_CR42","doi-asserted-by":"crossref","unstructured":"Hou, L. et al. Patch-based convolutional neural network for whole slide tissue image classification. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2424\u20132433 (IEEE, 2016).","DOI":"10.1109\/CVPR.2016.266"},{"key":"1411_CR43","doi-asserted-by":"crossref","unstructured":"Feng, J. & Zhou, Z.-H. Deep MIML Network. In 2017 Proceedings of the AAAI conference on artificial intelligence 1884\u20131890 (AAAI Press, 2017).","DOI":"10.1609\/aaai.v31i1.10890"},{"key":"1411_CR44","doi-asserted-by":"publisher","first-page":"1301","DOI":"10.1038\/s41591-019-0508-1","volume":"25","author":"G Campanella","year":"2019","unstructured":"Campanella, G. et al. Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nat. Med. 25, 1301\u20131309 (2019).","journal-title":"Nat. Med."},{"key":"1411_CR45","doi-asserted-by":"crossref","unstructured":"Xu, G. et al. Camel: a weakly supervised learning framework for histopathology image segmentation. In 2019 IEEE\/CVF International Conference on Computer Vision (ICCV) 10682\u201310691 (IEEE, 2019).","DOI":"10.1109\/ICCV.2019.01078"},{"key":"1411_CR46","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-020-66333-x","volume":"10","author":"F Kanavati","year":"2020","unstructured":"Kanavati, F. et al. Weakly-supervised learning for lung carcinoma classification using deep learning. Sci. Rep. 10, 9297 (2020).","journal-title":"Sci. Rep."},{"key":"1411_CR47","doi-asserted-by":"crossref","unstructured":"Zhu, W., Lou, Q., Vang, Y. S. & Xie, X. Deep multi-instance networks with sparse label assignment for whole mammogram classification. In Medical Image Computing and Computer Assisted Intervention \u2212 MICCAI 2017 603\u2013611 (Springer, 2017).","DOI":"10.1007\/978-3-319-66179-7_69"},{"key":"1411_CR48","doi-asserted-by":"crossref","unstructured":"Li, B., Li, Y. & Eliceiri, K. W. Dual-stream multiple instance learning network for whole slide image classification with self-supervised contrastive learning. In 2021 Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition (CVPR) 14318\u201314328 (IEEE, 2021).","DOI":"10.1109\/CVPR46437.2021.01409"}],"container-title":["npj Digital Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41746-024-01411-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-024-01411-2","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-024-01411-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,12,30]],"date-time":"2024-12-30T11:03:57Z","timestamp":1735556637000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41746-024-01411-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,12,30]]},"references-count":48,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2024,12]]}},"alternative-id":["1411"],"URL":"https:\/\/doi.org\/10.1038\/s41746-024-01411-2","relation":{},"ISSN":["2398-6352"],"issn-type":[{"value":"2398-6352","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,12,30]]},"assertion":[{"value":"22 April 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"22 December 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"30 December 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"383"}}