{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,27]],"date-time":"2026-08-27T15:04:46Z","timestamp":1787843086607,"version":"build-2784847793"},"reference-count":48,"publisher":"Public Library of Science (PLoS)","issue":"9","license":[{"start":{"date-parts":[[2025,9,30]],"date-time":"2025-09-30T00:00:00Z","timestamp":1759190400000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100010665","name":"H2020 Marie Sk\u0142odowska-Curie Actions","doi-asserted-by":"publisher","award":["860627"],"award-info":[{"award-number":["860627"]}],"id":[{"id":"10.13039\/100010665","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100010665","name":"H2020 Marie Sk\u0142odowska-Curie Actions","doi-asserted-by":"publisher","award":["860627"],"award-info":[{"award-number":["860627"]}],"id":[{"id":"10.13039\/100010665","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100010665","name":"H2020 Marie Sk\u0142odowska-Curie Actions","doi-asserted-by":"publisher","award":["860627"],"award-info":[{"award-number":["860627"]}],"id":[{"id":"10.13039\/100010665","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["www.plosdigitalhealth.org"],"crossmark-restriction":false},"short-container-title":["PLOS Digit Health"],"abstract":"<jats:p>We present a pioneering investigation into the application of deep learning techniques to analyze histopathological images for addressing the substantial challenge of automated prognostic prediction. Prognostic prediction poses a unique challenge as the ground truth labels are inherently weak, and the model must anticipate future events that are not directly observable in the image. To address this challenge, we propose a novel three-part framework comprising of a convolutional network based tissue segmentation algorithm for region of interest delineation, a contrastive learning module for feature extraction, and a nested multiple instance learning classification module. Our study explores the significance of various regions of interest within the histopathological slides and exploits diverse learning methods in real-world clinical scenarios. The pipeline is initially validated on artificially generated data and a simpler diagnostic task. Transitioning to prognostic prediction, tasks become more challenging. Employing bladder cancer as use case, our best models yield an AUC of 0.721 and 0.678 for recurrence and treatment outcome prediction respectively for a private data cohort. Altogether, this research serves as an initial investigation on the shortcomings of histopathological image analysis for treatment outcome prediction.<\/jats:p>","DOI":"10.1371\/journal.pdig.0000972","type":"journal-article","created":{"date-parts":[[2025,9,30]],"date-time":"2025-09-30T17:50:42Z","timestamp":1759254642000},"page":"e0000972","update-policy":"https:\/\/doi.org\/10.1371\/journal.pdig.corrections_policy","source":"Crossref","is-referenced-by-count":4,"title":["Self-contrastive weakly supervised learning framework for prognostic prediction using whole slide images"],"prefix":"10.1371","volume":"4","author":[{"ORCID":"https:\/\/orcid.org\/0009-0002-0285-6489","authenticated-orcid":true,"given":"Saul","family":"Fuster","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Farbod","family":"Khoraminia","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Julio","family":"Silva-Rodr\u00edguez","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Umay","family":"Kiraz","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Geert J. L. H.","family":"van Leenders","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Trygve","family":"Eftest\u00f8l","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Valery","family":"Naranjo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7760-5396","authenticated-orcid":true,"given":"Emiel A. M.","family":"Janssen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tahlita C. M.","family":"Zuiverloon","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kjersti","family":"Engan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2025,9,30]]},"reference":[{"issue":"5","key":"pdig.0000972.ref001","doi-asserted-by":"crossref","first-page":"775","DOI":"10.1038\/s41591-021-01343-4","article-title":"Deep learning in histopathology: the path to the clinic","volume":"27","author":"J van der Laak","year":"2021","journal-title":"Nat Med."},{"key":"pdig.0000972.ref002","doi-asserted-by":"crossref","first-page":"170","DOI":"10.1016\/j.media.2016.06.037","article-title":"Image analysis and machine learning in digital pathology: challenges and opportunities","volume":"33","author":"A Madabhushi","year":"2016","journal-title":"Med Image Anal."},{"issue":"1","key":"pdig.0000972.ref003","doi-asserted-by":"crossref","first-page":"75","DOI":"10.1016\/j.eururo.2021.08.010","article-title":"European association of urology guidelines on non-muscle-invasive bladder cancer (Ta, T1, and carcinoma in situ)","volume":"81","author":"M Babjuk","year":"2022","journal-title":"Eur Urol."},{"key":"pdig.0000972.ref004","doi-asserted-by":"crossref","unstructured":"Fuster S, Eftest\u00f8l T, Engan K. Nested multiple instance learning with attention mechanisms. In: 2022 21st IEEE International Conference on Machine Learning and Applications (ICMLA). 2022. p. 220\u20135.","DOI":"10.1109\/ICMLA55696.2022.00038"},{"issue":"2","key":"pdig.0000972.ref005","doi-asserted-by":"crossref","first-page":"234","DOI":"10.1016\/j.eururo.2012.07.033","article-title":"Epidemiology and risk factors of urothelial bladder cancer","volume":"63","author":"M Burger","year":"2013","journal-title":"Eur Urol."},{"issue":"16","key":"pdig.0000972.ref006","doi-asserted-by":"crossref","first-page":"1935","DOI":"10.1200\/JCO.2015.64.4070","article-title":"Definitions, end points, and clinical trial designs for non\u2013muscle-invasive bladder cancer: recommendations from the International Bladder Cancer Group","volume":"34","author":"AM Kamat","year":"2016","journal-title":"Journal of Clinical Oncology."},{"issue":"4","key":"pdig.0000972.ref007","doi-asserted-by":"crossref","first-page":"412","DOI":"10.1038\/s41374-020-00514-0","article-title":"Artificial intelligence and computational pathology","volume":"101","author":"M Cui","year":"2021","journal-title":"Lab Invest."},{"key":"pdig.0000972.ref008","doi-asserted-by":"crossref","first-page":"100135","DOI":"10.1016\/j.jpi.2022.100135","article-title":"Bladder cancer prognosis using deep neural networks and histopathology images","volume":"13","author":"W Barrios","year":"2022","journal-title":"J Pathol Inform."},{"key":"pdig.0000972.ref009","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1016\/j.csbj.2014.11.005","article-title":"Machine learning applications in cancer prognosis and prediction","volume":"13","author":"K Kourou","year":"2014","journal-title":"Comput Struct Biotechnol J."},{"issue":"1","key":"pdig.0000972.ref010","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1016\/j.urolonc.2007.07.010","article-title":"The impact of variant histology on the outcome of bladder cancer treated with curative intent","volume":"27","author":"PC Black","year":"2009","journal-title":"Urol Oncol."},{"issue":"23","key":"pdig.0000972.ref011","doi-asserted-by":"crossref","first-page":"5807","DOI":"10.3390\/cancers14235807","article-title":"Accurate diagnosis and survival prediction of bladder cancer using deep learning on histological slides","volume":"14","author":"Q Zheng","year":"2022","journal-title":"Cancers."},{"issue":"4","key":"pdig.0000972.ref012","doi-asserted-by":"crossref","first-page":"533","DOI":"10.1038\/s41379-021-00955-y","article-title":"Prediction of non-muscle invasive bladder cancer recurrence using machine learning of quantitative nuclear features","volume":"35","author":"N Tokuyama","year":"2022","journal-title":"Mod Pathol."},{"issue":"697","key":"pdig.0000972.ref013","doi-asserted-by":"crossref","DOI":"10.1126\/scitranslmed.abn4118","article-title":"Non-muscle-invasive bladder cancer molecular subtypes predict differential response to intravesical Bacillus Calmette-Gu\u00e9rin","volume":"15","author":"FC de Jong","year":"2023","journal-title":"Sci Transl Med."},{"issue":"9","key":"pdig.0000972.ref014","doi-asserted-by":"crossref","first-page":"3950","DOI":"10.1109\/TCYB.2019.2935141","article-title":"Weakly supervised deep learning for whole slide lung cancer image analysis","volume":"50","author":"X Wang","year":"2020","journal-title":"IEEE Trans Cybern."},{"key":"pdig.0000972.ref015","doi-asserted-by":"crossref","first-page":"101861","DOI":"10.1016\/j.compmedimag.2021.101861","article-title":"Histopathology classification and localization of colorectal cancer using global labels by weakly supervised deep learning","volume":"88","author":"C Zhou","year":"2021","journal-title":"Comput Med Imaging Graph."},{"key":"pdig.0000972.ref016","doi-asserted-by":"crossref","first-page":"102093","DOI":"10.1016\/j.compmedimag.2022.102093","article-title":"Weakly supervised deep learning for prediction of treatment effectiveness on ovarian cancer from histopathology images","volume":"99","author":"C-W Wang","year":"2022","journal-title":"Comput Med Imaging Graph."},{"key":"pdig.0000972.ref017","doi-asserted-by":"crossref","unstructured":"Andreassen C, Fuster S, Hardardottir H, Janssen EAM, Engan K. Deep learning for predicting metastasis on melanoma WSIs. In: 2023 IEEE 20th International Symposium on Biomedical Imaging (ISBI). IEEE; 2023. p. 1\u20135.","DOI":"10.1109\/ISBI53787.2023.10230474"},{"key":"pdig.0000972.ref018","first-page":"1","article-title":"Probabilistic attention based on gaussian processes for deep multiple instance learning","author":"A Schmidt","year":"2023","journal-title":"IEEE Transactions on Neural Networks and Learning Systems."},{"key":"pdig.0000972.ref019","first-page":"11226","article-title":"SI-MIL: taming deep MIL for self-interpretability in gigapixel histopathology","volume":"2024","author":"S Kapse","year":"2024","journal-title":"Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit."},{"key":"pdig.0000972.ref020","unstructured":"Ilse M, Tomczak J, Welling M. Attention-based deep multiple instance learning. In: International Conference on Machine Learning. PMLR; 2018. p. 2127\u201336."},{"issue":"6","key":"pdig.0000972.ref021","doi-asserted-by":"crossref","first-page":"555","DOI":"10.1038\/s41551-020-00682-w","article-title":"Data-efficient and weakly supervised computational pathology on whole-slide images","volume":"5","author":"MY Lu","year":"2021","journal-title":"Nat Biomed Eng."},{"key":"pdig.0000972.ref022","doi-asserted-by":"crossref","first-page":"101813","DOI":"10.1016\/j.media.2020.101813","article-title":"Deep neural network models for computational histopathology: a survey","volume":"67","author":"CL Srinidhi","year":"2021","journal-title":"Med Image Anal."},{"key":"pdig.0000972.ref023","doi-asserted-by":"crossref","unstructured":"Zhang H, Meng Y, Zhao Y, Qiao Y, Yang X, Coupland SE. DtfD-MIL: double-tier feature distillation multiple instance learning for histopathology whole slide image classification. In: Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 2022. p. 18802\u201312.","DOI":"10.1109\/CVPR52688.2022.01824"},{"issue":"8","key":"pdig.0000972.ref024","doi-asserted-by":"crossref","first-page":"1301","DOI":"10.1038\/s41591-019-0508-1","article-title":"Clinical-grade computational pathology using weakly supervised deep learning on whole slide images","volume":"25","author":"G Campanella","year":"2019","journal-title":"Nat Med."},{"issue":"5","key":"pdig.0000972.ref025","doi-asserted-by":"crossref","first-page":"236","DOI":"10.1038\/s42256-019-0052-1","article-title":"Pathologist-level interpretable whole-slide cancer diagnosis with deep learning","volume":"1","author":"Z Zhang","year":"2019","journal-title":"Nature Machine Intelligence."},{"key":"pdig.0000972.ref026","doi-asserted-by":"crossref","first-page":"115813","DOI":"10.1109\/ACCESS.2021.3104724","article-title":"Automatic diagnostic tool for predicting cancer grade in bladder cancer patients using deep learning","volume":"9","author":"R Wetteland","year":"2021","journal-title":"IEEE Access."},{"key":"pdig.0000972.ref027","doi-asserted-by":"crossref","first-page":"101789","DOI":"10.1016\/j.media.2020.101789","article-title":"Whole slide images based cancer survival prediction using attention guided deep multiple instance learning networks","volume":"65","author":"J Yao","year":"2020","journal-title":"Med Image Anal."},{"key":"pdig.0000972.ref028","first-page":"3","volume-title":"MICCAI Workshop on Cancer Prevention through Early Detection","author":"VL Le","year":"2023"},{"key":"pdig.0000972.ref029","doi-asserted-by":"crossref","first-page":"103020","DOI":"10.1016\/j.media.2023.103020","article-title":"AdvMIL: adversarial multiple instance learning for the survival analysis on whole-slide images","volume":"91","author":"P Liu","year":"2024","journal-title":"Med Image Anal."},{"key":"pdig.0000972.ref030","doi-asserted-by":"crossref","first-page":"120280","DOI":"10.1016\/j.eswa.2023.120280","article-title":"DSCA: a dual-stream network with cross-attention on whole-slide image pyramids for cancer prognosis","volume":"227","author":"P Liu","year":"2023","journal-title":"Expert Systems with Applications."},{"key":"pdig.0000972.ref031","doi-asserted-by":"crossref","first-page":"103097","DOI":"10.1016\/j.media.2024.103097","article-title":"Immune subtyping of melanoma whole slide images using multiple instance learning","volume":"93","author":"L Godson","year":"2024","journal-title":"Med Image Anal."},{"key":"pdig.0000972.ref032","unstructured":"Chen T, Kornblith S, Norouzi M, Hinton G. A simple framework for contrastive learning of visual representations. In: International Conference on Machine Learning. PMLR; 2020. p. 1597\u2013607."},{"key":"pdig.0000972.ref033","doi-asserted-by":"crossref","first-page":"100198","DOI":"10.1016\/j.mlwa.2021.100198","article-title":"Self supervised contrastive learning for digital histopathology","volume":"7","author":"O Ciga","year":"2022","journal-title":"Machine Learning with Applications."},{"key":"pdig.0000972.ref034","doi-asserted-by":"crossref","first-page":"100133","DOI":"10.1016\/j.jpi.2022.100133","article-title":"A self-supervised contrastive learning approach for whole slide image representation in digital pathology","volume":"13","author":"PA Fashi","year":"2022","journal-title":"J Pathol Inform."},{"key":"pdig.0000972.ref035","doi-asserted-by":"crossref","unstructured":"Ke J, Shen Y, Liang X, Shen D. Contrastive learning based stain normalization across multiple tumor in histopathology. In: Medical Image Computing and Computer Assisted Intervention\u2013MICCAI 2021 : 24th International Conference, Strasbourg, France, September 27\u2013October 1, 2021, Proceedings, Part VIII. 2021. p. 571\u201380.","DOI":"10.1007\/978-3-030-87237-3_55"},{"issue":"7","key":"pdig.0000972.ref036","doi-asserted-by":"crossref","first-page":"202","DOI":"10.3390\/jimaging8070202","article-title":"StainCUT: stain normalization with contrastive learning","volume":"8","author":"JC Guti\u00e9rrez P\u00e9rez","year":"2022","journal-title":"J Imaging."},{"key":"pdig.0000972.ref037","doi-asserted-by":"crossref","unstructured":"Li B, Li Y, Eliceiri KW. Dual-stream multiple instance learning network for whole slide image classification with self-supervised contrastive learning. Conf Comput Vis Pattern Recognit Workshops. 2021;2021:14318\u201328. https:\/\/doi.org\/10.1109\/CVPR46437.2021.01409 35047230","DOI":"10.1109\/CVPR46437.2021.01409"},{"key":"pdig.0000972.ref038","doi-asserted-by":"crossref","unstructured":"Li J, Zheng Y, Wu K, Shi J, Xie F, Jiang Z. Lesion-aware contrastive representation learning for histopathology whole slide images analysis. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer; 2022. p. 273\u201382.","DOI":"10.1007\/978-3-031-16434-7_27"},{"key":"pdig.0000972.ref039","first-page":"29484","article-title":"Dual-curriculum contrastive multi-instance learning for cancer prognosis analysis with whole slide images","volume":"35","author":"C Tu","year":"2022","journal-title":"Advances in Neural Information Processing Systems."},{"key":"pdig.0000972.ref040","doi-asserted-by":"crossref","first-page":"1533033820946787","DOI":"10.1177\/1533033820946787","article-title":"A multiscale approach for whole-slide image segmentation of five tissue classes in urothelial carcinoma slides","volume":"19","author":"R Wetteland","year":"2020","journal-title":"Technology in Cancer Research & Treatment."},{"key":"pdig.0000972.ref041","doi-asserted-by":"crossref","unstructured":"Fuster S, Khoraminia F, Eftest\u00f8l T, Zuiverloon T, Engan K. Active learning based domain adaptation for tissue segmentation of histopathological images. In:2023 31st European Signal Processing Conference (EUSIPCO). IEEE; 2023. p. 1045\u20139.","DOI":"10.23919\/EUSIPCO58844.2023.10290058"},{"issue":"1","key":"pdig.0000972.ref042","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1002\/nau.22465","article-title":"Lamina propria: the functional center of the bladder?","volume":"33","author":"K-E Andersson","year":"2014","journal-title":"Neurourol Urodyn."},{"issue":"50","key":"pdig.0000972.ref043","doi-asserted-by":"crossref","first-page":"82609","DOI":"10.18632\/oncotarget.12089","article-title":"Positive association of collagen type I with non-muscle invasive bladder cancer progression","volume":"7","author":"M Brooks","year":"2016","journal-title":"Oncotarget."},{"key":"pdig.0000972.ref044","doi-asserted-by":"crossref","unstructured":"Wetteland R, Engan K, Eftesol T. Parameterized extraction of tiles in multilevel gigapixel images. In: 2021 12th International Symposium on Image and Signal Processing and Analysis (ISPA). 2021. p. 78\u201383.","DOI":"10.1109\/ISPA52656.2021.9552104"},{"key":"pdig.0000972.ref045","first-page":"18661","article-title":"Supervised contrastive learning","volume":"33","author":"P Khosla","year":"2020","journal-title":"Advances in Neural Information Processing Systems."},{"key":"pdig.0000972.ref046","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1016\/S0004-3702(96)00034-3","article-title":"Solving the multiple instance problem with axis-parallel rectangles","volume":"89","author":"TG Dietterich","year":"1997","journal-title":"Artificial Intelligence."},{"key":"pdig.0000972.ref047","article-title":"Imagenet classification with deep convolutional neural networks","volume":"25","author":"A Krizhevsky","year":"2012","journal-title":"Advances in neural information processing systems."},{"key":"pdig.0000972.ref048","doi-asserted-by":"crossref","unstructured":"Abraham N, Khan NM. A novel focal tversky loss function with improved attention u-net for lesion segmentation. In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019). 2019. p. 683\u20137.","DOI":"10.1109\/ISBI.2019.8759329"}],"container-title":["PLOS Digital Health"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pdig.0000972","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,30]],"date-time":"2025-09-30T17:50:51Z","timestamp":1759254651000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pdig.0000972"}},"subtitle":[],"editor":[{"given":"Sebasti\u00e1n Andr\u00e9s","family":"Cajas","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2025,9,30]]},"references-count":48,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2025,9,30]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pdig.0000972","relation":{},"ISSN":["2767-3170"],"issn-type":[{"value":"2767-3170","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,9,30]]}}}