{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,8]],"date-time":"2026-04-08T18:37:04Z","timestamp":1775673424944,"version":"3.50.1"},"reference-count":45,"publisher":"Oxford University Press (OUP)","issue":"14","license":[{"start":{"date-parts":[[2022,6,8]],"date-time":"2022-06-08T00:00:00Z","timestamp":1654646400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"DOI":"10.13039\/100000054","name":"National Cancer Institute","doi-asserted-by":"publisher","award":["4UH3-CA225021-03"],"award-info":[{"award-number":["4UH3-CA225021-03"]}],"id":[{"id":"10.13039\/100000054","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000054","name":"National Cancer Institute","doi-asserted-by":"publisher","award":["1U24CA180924-01A1"],"award-info":[{"award-number":["1U24CA180924-01A1"]}],"id":[{"id":"10.13039\/100000054","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000054","name":"National Cancer Institute","doi-asserted-by":"publisher","award":["3U24CA215109-02"],"award-info":[{"award-number":["3U24CA215109-02"]}],"id":[{"id":"10.13039\/100000054","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000054","name":"National Cancer Institute","doi-asserted-by":"publisher","award":["1UG3CA225021-01"],"award-info":[{"award-number":["1UG3CA225021-01"]}],"id":[{"id":"10.13039\/100000054","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2022,7,11]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Motivation<\/jats:title><jats:p>Whole slide tissue images contain detailed data on the sub-cellular structure of cancer. Quantitative analyses of this data can lead to novel biomarkers for better cancer diagnosis and prognosis and can improve our understanding of cancer mechanisms. Such analyses are challenging to execute because of the sizes and complexity of whole slide image data and relatively limited volume of training data for machine learning methods.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>We propose and experimentally evaluate a multi-resolution deep learning method for breast cancer survival analysis. The proposed method integrates image data at multiple resolutions and tumor, lymphocyte and nuclear segmentation results from deep learning models. Our results show that this approach can significantly improve the deep learning model performance compared to using only the original image data. The proposed approach achieves a c-index value of 0.706 compared to a c-index value of 0.551 from an approach that uses only color image data at the highest image resolution. Furthermore, when clinical features (sex, age and cancer stage) are combined with image data, the proposed approach achieves a c-index of 0.773.<\/jats:p><\/jats:sec><jats:sec><jats:title>Availability and implementation<\/jats:title><jats:p>https:\/\/github.com\/SBU-BMI\/deep_survival_analysis<\/jats:p><\/jats:sec>","DOI":"10.1093\/bioinformatics\/btac381","type":"journal-article","created":{"date-parts":[[2022,6,8]],"date-time":"2022-06-08T12:01:43Z","timestamp":1654689703000},"page":"3629-3637","source":"Crossref","is-referenced-by-count":24,"title":["Deep learning for survival analysis in breast cancer with whole slide image data"],"prefix":"10.1093","volume":"38","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3833-9475","authenticated-orcid":false,"given":"Huidong","family":"Liu","sequence":"first","affiliation":[{"name":"Department of Computer Science, Stony Brook University , Stony Brook, NY 11794, USA"}]},{"given":"Tahsin","family":"Kurc","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Stony Brook University , Stony Brook, NY 11794, USA"}]}],"member":"286","published-online":{"date-parts":[[2022,6,8]]},"reference":[{"key":"2023041405363964200_","first-page":"480","volume-title":"International Conference on Medical Image Computing and Computer-Assisted Intervention","author":"Abbet","year":"2020"},{"key":"2023041405363964200_","author":"Abousamra","year":"2019"},{"key":"2023041405363964200_","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41523-020-0154-2","article-title":"Report on computational assessment of tumor infiltrating lymphocytes from the international immuno-oncology biomarker working group","volume":"6","author":"Amgad","year":"2020","journal-title":"NPJ Breast Cancer"},{"key":"2023041405363964200_","doi-asserted-by":"crossref","first-page":"512","DOI":"10.1002\/path.5028","article-title":"Pancancer insights from the cancer genome atlas: the pathologist\u2019s perspective","volume":"244","author":"Cooper","year":"2018","journal-title":"J. Pathol"},{"key":"2023041405363964200_","doi-asserted-by":"crossref","first-page":"1519","DOI":"10.1038\/s41591-019-0583-3","article-title":"Deep learning-based classification of mesothelioma improves prediction of patient outcome","volume":"25","author":"Courtiol","year":"2019","journal-title":"Nat. Med"},{"key":"2023041405363964200_","author":"Davidson-Pilon","year":"2019"},{"key":"2023041405363964200_","first-page":"248","author":"Deng","year":"2009"},{"key":"2023041405363964200_","doi-asserted-by":"crossref","first-page":"470","DOI":"10.1007\/s11684-020-0782-9","article-title":"Deep learning in digital pathology image analysis: a survey","volume":"14","author":"Deng","year":"2020","journal-title":"Front. Med"},{"key":"2023041405363964200_","first-page":"428","author":"Di","year":"2020"},{"key":"2023041405363964200_","doi-asserted-by":"crossref","first-page":"264","DOI":"10.3389\/fmed.2019.00264","article-title":"Deep learning for whole slide image analysis: an overview","volume":"6","author":"Dimitriou","year":"2019","journal-title":"Front. Med. (Lausanne)"},{"key":"2023041405363964200_","doi-asserted-by":"crossref","first-page":"101563","DOI":"10.1016\/j.media.2019.101563","article-title":"Hover-net: simultaneous segmentation and classification of nuclei in multi-tissue histology images","volume":"58","author":"Graham","year":"2019","journal-title":"Med. Image Anal"},{"key":"2023041405363964200_","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1016\/j.ymeth.2014.06.015","article-title":"Digital pathology and image analysis in tissue biomarker research","volume":"70","author":"Hamilton","year":"2014","journal-title":"Methods"},{"key":"2023041405363964200_","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s12920-019-0624-2","article-title":"Interpretable deep neural network for cancer survival analysis by integrating genomic and clinical data","volume":"12","author":"Hao","year":"2019","journal-title":"BMC Med. Genomics"},{"key":"2023041405363964200_","first-page":"8533","author":"Hou","year":"2019"},{"key":"2023041405363964200_","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41597-020-0528-1","article-title":"Dataset of segmented nuclei in hematoxylin and eosin stained histopathology images of ten cancer types","volume":"7","author":"Hou","year":"2020","journal-title":"Sci. Data"},{"key":"2023041405363964200_","doi-asserted-by":"crossref","first-page":"134","DOI":"10.1016\/j.patcog.2018.05.014","article-title":"Deep learning for image-based cancer detection and diagnosis\u2014a survey","volume":"83","author":"Hu","year":"2018","journal-title":"Pattern Recognition"},{"key":"2023041405363964200_","first-page":"2127","author":"Ilse","year":"2018"},{"key":"2023041405363964200_","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1186\/s13058-020-1248-3","article-title":"A deep learning image-based intrinsic molecular subtype classifier of breast tumors reveals tumor heterogeneity that may affect survival","volume":"22","author":"Jaber","year":"2020","journal-title":"Breast Cancer Res"},{"key":"2023041405363964200_","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.artmed.2019.06.001","article-title":"A deep survival analysis method based on ranking","volume":"98","author":"Jing","year":"2019","journal-title":"Artif. Intell. Med"},{"key":"2023041405363964200_","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s12874-018-0482-1","article-title":"Deepsurv: personalized treatment recommender system using a cox proportional hazards deep neural network","volume":"18","author":"Katzman","year":"2018","journal-title":"BMC Med. Res. Methodol"},{"key":"2023041405363964200_","doi-asserted-by":"crossref","first-page":"511","DOI":"10.1007\/s00428-018-2485-z","article-title":"Precision immunoprofiling by image analysis and artificial intelligence","volume":"474","author":"Koelzer","year":"2019","journal-title":"Virchows Arch"},{"key":"2023041405363964200_","doi-asserted-by":"crossref","first-page":"2845","DOI":"10.1109\/TMI.2021.3056023","article-title":"Self-path: self-supervision for classification of pathology images with limited annotations","volume":"40","author":"Koohbanani","year":"2021","journal-title":"IEEE Trans. Med. Imaging"},{"key":"2023041405363964200_","doi-asserted-by":"crossref","first-page":"1491","DOI":"10.1016\/j.ajpath.2020.03.012","article-title":"Utilizing automated breast cancer detection to identify spatial distributions of tumor-infiltrating lymphocytes in invasive breast cancer","volume":"190","author":"Le","year":"2020","journal-title":"Am. J. Pathol"},{"key":"2023041405363964200_","first-page":"174","author":"Li","year":"2018"},{"key":"2023041405363964200_","first-page":"1715","author":"Li","year":"2016"},{"key":"2023041405363964200_","doi-asserted-by":"crossref","first-page":"60","DOI":"10.1016\/j.media.2017.07.005","article-title":"A survey on deep learning in medical image analysis","volume":"42","author":"Litjens","year":"2017","journal-title":"Med. Image Anal"},{"key":"2023041405363964200_","doi-asserted-by":"crossref","first-page":"1438","DOI":"10.1038\/s41374-018-0095-7","article-title":"Nuclear shape and orientation features from H&E images predict survival in early-stage estrogen receptor-positive breast cancers","volume":"98","author":"Lu","year":"2018","journal-title":"Lab. Invest"},{"key":"2023041405363964200_","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1016\/j.celrep.2018.03.086","article-title":"Spatial organization and molecular correlation of tumor-infiltrating lymphocytes using deep learning on pathology images","volume":"23","author":"Saltz","year":"2018","journal-title":"Cell Rep"},{"key":"2023041405363964200_","first-page":"4510","author":"Sandler","year":"2018"},{"key":"2023041405363964200_","first-page":"101813","author":"Srinidhi","year":"2020"},{"key":"2023041405363964200_","first-page":"1","author":"Szegedy","year":"2015"},{"key":"2023041405363964200_","doi-asserted-by":"crossref","first-page":"812","DOI":"10.1016\/j.immuni.2018.03.023","article-title":"The immune landscape of cancer","volume":"48","author":"Thorsson","year":"2018","journal-title":"Immunity"},{"key":"2023041405363964200_","first-page":"26","article-title":"Lecture 6.5-rmsprop: divide the gradient by a running average of its recent magnitude","volume":"4","author":"Tieleman","year":"2012","journal-title":"COURSERA Neural Netw. Mach. Learn"},{"key":"2023041405363964200_","doi-asserted-by":"crossref","first-page":"53","DOI":"10.3389\/fbioe.2019.00053","article-title":"Methods for segmentation and classification of digital microscopy tissue images","volume":"7","author":"Vu","year":"2019","journal-title":"Front. Bioeng. Biotechnol"},{"key":"2023041405363964200_","first-page":"1","article-title":"Prediction of recurrence in early stage non-small cell lung cancer using computer extracted nuclear features from digital H&E images","volume":"7","author":"Wang","year":"2017","journal-title":"Sci. Rep"},{"key":"2023041405363964200_","doi-asserted-by":"crossref","first-page":"e0233678","DOI":"10.1371\/journal.pone.0233678","article-title":"Deep learning-based survival prediction for multiple cancer types using histopathology images","volume":"15","author":"Wulczyn","year":"2020","journal-title":"PLoS One"},{"key":"2023041405363964200_","first-page":"449","author":"Xiao","year":"2020"},{"key":"2023041405363964200_","doi-asserted-by":"crossref","first-page":"4550","DOI":"10.1109\/TNNLS.2017.2766168","article-title":"Deep learning in microscopy image analysis: a survey","volume":"29","author":"Xing","year":"2017","journal-title":"IEEE Trans. Neural Netw. Learning Syst"},{"key":"2023041405363964200_","doi-asserted-by":"crossref","first-page":"214","DOI":"10.1016\/j.neucom.2016.01.034","article-title":"A deep convolutional neural network for segmenting and classifying epithelial and stromal regions in histopathological images","volume":"191","author":"Xu","year":"2016","journal-title":"Neurocomputing"},{"key":"2023041405363964200_","first-page":"649","volume-title":"International Conference on Medical Image Computing and Computer-Assisted Intervention","author":"Yao","year":"2016"},{"key":"2023041405363964200_","first-page":"406","volume-title":"International Conference on Medical Image Computing and Computer-Assisted Intervention","author":"Yao","year":"2017"},{"key":"2023041405363964200_","first-page":"496","volume-title":"International Conference on Medical Image Computing and Computer-Assisted Intervention","author":"Yao","year":"2019"},{"key":"2023041405363964200_","doi-asserted-by":"crossref","first-page":"101789","DOI":"10.1016\/j.media.2020.101789","article-title":"Whole slide images based cancer survival prediction using attention guided deep multiple instance learning networks","volume":"65","author":"Yao","year":"2020","journal-title":"Med. Image Anal"},{"key":"2023041405363964200_","first-page":"544","author":"Zhu","year":"2016"},{"key":"2023041405363964200_","first-page":"7234","author":"Zhu","year":"2017"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/advance-article-pdf\/doi\/10.1093\/bioinformatics\/btac381\/44072032\/btac381.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/38\/14\/3629\/49883962\/btac381.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/38\/14\/3629\/49883962\/btac381.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,22]],"date-time":"2023-11-22T20:48:06Z","timestamp":1700686086000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/38\/14\/3629\/6604265"}},"subtitle":[],"editor":[{"given":"Olga","family":"Vitek","sequence":"additional","affiliation":[]}],"short-title":[],"issued":{"date-parts":[[2022,6,8]]},"references-count":45,"journal-issue":{"issue":"14","published-print":{"date-parts":[[2022,7,11]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btac381","relation":{},"ISSN":["1367-4803","1367-4811"],"issn-type":[{"value":"1367-4803","type":"print"},{"value":"1367-4811","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2022,7,15]]},"published":{"date-parts":[[2022,6,8]]}}}