{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,24]],"date-time":"2026-10-24T07:56:49Z","timestamp":1792828609114,"version":"4.5.0"},"reference-count":33,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2019,5,15]],"date-time":"2019-05-15T00:00:00Z","timestamp":1557878400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2019,5,15]],"date-time":"2019-05-15T00:00:00Z","timestamp":1557878400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Sci Rep"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>In the context of precision medicine with immunotherapies there is an increasing need for companion diagnostic tests to identify potential therapy responders and avoid treatment coming along with severe adverse events for non-responders. Here, we present a retrospective case study to discover image-based signatures for developing a potential companion diagnostic test for ipilimumab (IPI) in malignant melanoma. Signature discovery is based on digital pathology and fully automatic quantitative image analysis using virtual multiplexing as well as machine learning and deep learning on whole-slide images. We systematically correlated the patient outcome data with potentially relevant local image features using a Tissue Phenomics approach with a sound cross validation procedure for reliable performance evaluation. Besides uni-variate models we also studied combinations of signatures in several multi-variate models. The most robust and best performing model was a decision tree model based on relative densities of CD8+ tumor infiltrating lymphocytes in the intra-tumoral infiltration region. Our results are well in agreement with observations described in previously published studies regarding the predictive value of the immune contexture, and thus, provide predictive potential for future development of a companion diagnostic test.<\/jats:p>","DOI":"10.1038\/s41598-019-43525-8","type":"journal-article","created":{"date-parts":[[2019,5,15]],"date-time":"2019-05-15T10:03:46Z","timestamp":1557914626000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":61,"title":["Automatic discovery of image-based signatures for ipilimumab response prediction in malignant melanoma"],"prefix":"10.1038","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6171-6462","authenticated-orcid":false,"given":"Nathalie","family":"Harder","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ralf","family":"Sch\u00f6nmeyer","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Katharina","family":"Nekolla","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Armin","family":"Meier","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nicolas","family":"Brieu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Carolina","family":"Vanegas","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gabriele","family":"Madonna","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mariaelena","family":"Capone","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gerardo","family":"Botti","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8322-475X","authenticated-orcid":false,"given":"Paolo A.","family":"Ascierto","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"G\u00fcnter","family":"Schmidt","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2019,5,15]]},"reference":[{"key":"43525_CR1","doi-asserted-by":"publisher","first-page":"23","DOI":"10.1056\/NEJMoa1504030","volume":"373","author":"J Larkin","year":"2015","unstructured":"Larkin, J. et al. Combined Nivolumab and Ipilimumab or Monotherapy in Previously Untreated Melanoma. N. Engl. J. Med. 373, 23\u201334 (2015).","journal-title":"N. Engl. J. Med."},{"key":"43525_CR2","doi-asserted-by":"publisher","first-page":"e542","DOI":"10.1016\/S1470-2045(16)30406-5","volume":"17","author":"GT Gibney","year":"2016","unstructured":"Gibney, G. T., Weiner, L. M. & Atkins, M. B. Predictive biomarkers for checkpoint inhibitor-based immunotherapy. Lancet Oncol. 17, e542\u2013e551 (2016).","journal-title":"Lancet Oncol."},{"key":"43525_CR3","first-page":"208","volume":"3","author":"C Teixid\u00f3","year":"2015","unstructured":"Teixid\u00f3, C., Gonz\u00e1lez-Cao, M., Karachaliou, N. & Rosell, R. Predictive factors for immunotherapy in melanoma. Ann. Transl. Med. 3, 208 (2015).","journal-title":"Ann. Transl. Med."},{"key":"43525_CR4","doi-asserted-by":"publisher","first-page":"490","DOI":"10.1111\/pcmr.12371","volume":"28","author":"A Lad\u00e1nyi","year":"2015","unstructured":"Lad\u00e1nyi, A. Prognostic and predictive significance of immune cells infiltrating cutaneous melanoma. Pigment Cell Melanoma Res. 28, 490\u2013500 (2015).","journal-title":"Pigment Cell Melanoma Res."},{"key":"43525_CR5","doi-asserted-by":"publisher","first-page":"389","DOI":"10.21037\/atm.2017.08.10","volume":"5","author":"E Espinosa","year":"2017","unstructured":"Espinosa, E. et al. Predictive factors of response to immunotherapy\u2014a review from the Spanish Melanoma Group (GEM). Ann. Transl. Med. 5, 389 (2017).","journal-title":"Ann. Transl. Med."},{"key":"43525_CR6","doi-asserted-by":"publisher","first-page":"1009","DOI":"10.1158\/1078-0432.CCR-12-2982","volume":"19","author":"PA Ascierto","year":"2013","unstructured":"Ascierto, P. A., Kalos, M., Schaer, D. A., Callahan, M. K. & Wolchok, J. D. Biomarkers for Immunostimulatory Monoclonal Antibodies in Combination Strategies for Melanoma and Other Tumor Types. Clin. Cancer Res. 19, 1009 (2013).","journal-title":"Clin. Cancer Res."},{"key":"43525_CR7","doi-asserted-by":"publisher","DOI":"10.1186\/1479-5876-9-204","volume":"9","author":"O Hamid","year":"2011","unstructured":"Hamid, O. et al. A prospective phase II trial exploring the association between tumor microenvironment biomarkers and clinical activity of ipilimumab in advanced melanoma. J. Transl. Med. 9, 204 (2011).","journal-title":"J. Transl. Med."},{"key":"43525_CR8","doi-asserted-by":"publisher","first-page":"1019","DOI":"10.1007\/s00262-011-1172-6","volume":"61","author":"R-R Ji","year":"2012","unstructured":"Ji, R.-R. et al. An immune-active tumor microenvironment favors clinical response to ipilimumab. Cancer Immunol. Immunother. 61, 1019\u20131031 (2012).","journal-title":"Cancer Immunol. Immunother."},{"key":"43525_CR9","doi-asserted-by":"publisher","first-page":"2189","DOI":"10.1056\/NEJMoa1406498","volume":"371","author":"A Snyder","year":"2014","unstructured":"Snyder, A. et al. Genetic Basis for Clinical Response to CTLA-4 Blockade in Melanoma. N. Engl. J. Med. 371, 2189\u20132199 (2014).","journal-title":"N. Engl. J. Med."},{"key":"43525_CR10","doi-asserted-by":"publisher","DOI":"10.1186\/1479-5876-10-205","volume":"10","author":"J Galon","year":"2012","unstructured":"Galon, J. et al. Cancer classification using the Immunoscore: a worldwide task force. J. Transl. Med. 10, 205 (2012).","journal-title":"J. Transl. Med."},{"key":"43525_CR11","doi-asserted-by":"publisher","first-page":"P193","DOI":"10.1186\/2051-1426-2-S3-P193","volume":"2","author":"M Capone","year":"2014","unstructured":"Capone, M. et al. Immunoscore: a new possible approach for melanoma classification. J. Immunother. Cancer 2, P193\u2013P193 (2014).","journal-title":"J. Immunother. Cancer"},{"key":"43525_CR12","doi-asserted-by":"publisher","DOI":"10.1186\/1479-5876-12-S1-P11","volume":"12","author":"C Bifulco","year":"2014","unstructured":"Bifulco, C. et al. MISIPI study: Melanoma ImmunoScore evaluation in patients treated with IPIlimumab. J. Transl. Med. 12, P11 (2014).","journal-title":"J. Transl. Med."},{"key":"43525_CR13","doi-asserted-by":"publisher","DOI":"10.1186\/s12967-016-1029-z","volume":"14","author":"J Galon","year":"2016","unstructured":"Galon, J. et al. Immunoscore and Immunoprofiling in cancer: an update from the melanoma and immunotherapy bridge 2015. J. Transl. Med. 14, 273 (2016).","journal-title":"J. Transl. Med."},{"issue":"12","key":"43525_CR14","doi-asserted-by":"publisher","first-page":"e1405206","DOI":"10.1080\/2162402X.2017.1405206","volume":"7","author":"Gabriele Madonna","year":"2018","unstructured":"Madonna, G. et al. PD-L1 expression with immune-infiltrate evaluation and outcome prediction in melanoma patients treated with ipilimumab. OncoImmunology e1405206, https:\/\/doi.org\/10.1080\/2162402X.2017.1405206 (2017).","journal-title":"OncoImmunology"},{"key":"43525_CR15","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-018-22564-7","volume":"8","author":"N Harder","year":"2018","unstructured":"Harder, N. et al. Tissue Phenomics for prognostic biomarker discovery in low- and intermediate-risk prostate cancer. Sci. Rep. 8, 4470 (2018).","journal-title":"Sci. Rep."},{"key":"43525_CR16","unstructured":"Binnig, G., Huss, R. & Schmidt, G. Tissue Phenomics: Profiling Cancer Patients for Treatment Decisions. (Pan Stanford Publishing, 2018)."},{"key":"43525_CR17","unstructured":"Harder, N. et al. Optimal Patient Selection for Immunotherapy in Melanoma using Tissue Phenomics. In Proc. 6th European Post-Chicago Melanoma\/Skin Cancer Meeting (2016)."},{"key":"43525_CR18","doi-asserted-by":"crossref","unstructured":"Brieu, N., Pauly, O., Zimmermann, J., Binnig, G. & Schmidt, G. Slide-specific models for segmentation of differently stained digital histopathology whole slide images. In Proc. SPIE Medical Imaging 2016: Image Processing (eds Styner, M. A. & Angelini, E. D.) 9784, 978410-978410\u20137 (SPIE, 2016).","DOI":"10.1117\/12.2208620"},{"key":"43525_CR19","unstructured":"Criminisi, A., Shotton, J. & Bucciarelli, S. Decision Forests with Long-Range Spatial Context for Organ Localization in CT Volumes. In MICCAI workshop on Probabilistic Models for Medical Image Analysis (MICCAI-PMMIA) 69\u201380 (2009)."},{"key":"43525_CR20","doi-asserted-by":"crossref","unstructured":"Yigitsoy, M., Schmidt, G. & Binnig, G. Hierarchical patch-based co-registration of differently stained histopathology slides. in Medical Imaging 2017: Digital Pathology, Proc. SPIE (2017).","DOI":"10.1117\/12.2254266"},{"key":"43525_CR21","doi-asserted-by":"publisher","first-page":"484","DOI":"10.1038\/nature16961","volume":"529","author":"D Silver","year":"2016","unstructured":"Silver, D. et al. Mastering the game of Go with deep neural networks and tree search. Nature 529, 484 (2016).","journal-title":"Nature"},{"key":"43525_CR22","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","volume":"115","author":"O Russakovsky","year":"2015","unstructured":"Russakovsky, O. et al. ImageNet Large Scale Visual Recognition Challenge. Int. J. Comput. Vis. IJCV 115, 211\u2013252 (2015).","journal-title":"Int. J. Comput. Vis. IJCV"},{"key":"43525_CR23","doi-asserted-by":"publisher","unstructured":"Szegedy, C. et al. Going deeper with convolutions. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 1\u20139, https:\/\/doi.org\/10.1109\/CVPR.2015.7298594 (IEEE, 2015)","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"43525_CR24","doi-asserted-by":"publisher","first-page":"57","DOI":"10.1186\/s40425-016-0161-x","volume":"4","author":"F Hermitte","year":"2016","unstructured":"Hermitte, F. Biomarkers immune monitoring technology primer: Immunoscore\u00ae Colon. J. Immunother. Cancer 4, 57\u201357 (2016).","journal-title":"J. Immunother. Cancer"},{"key":"43525_CR25","unstructured":"Breiman, L., Friedman, J., Stone, C. J. & Olshen, R. A. Classification and Regression Trees. (Taylor & Francis Ltd, 1984)."},{"key":"43525_CR26","doi-asserted-by":"crossref","unstructured":"Vapnik, V. N. The nature of statistical learning theory. (Springer, 1995).","DOI":"10.1007\/978-1-4757-2440-0"},{"key":"43525_CR27","doi-asserted-by":"publisher","first-page":"386","DOI":"10.1037\/h0042519","volume":"65","author":"F Rosenblatt","year":"1958","unstructured":"Rosenblatt, F. The perceptron: a probabilistic model for information storage and organization in the brain. Psychol. Rev. 65, 386\u2013408 (1958).","journal-title":"Psychol. Rev."},{"key":"43525_CR28","doi-asserted-by":"crossref","first-page":"215","DOI":"10.1111\/j.2517-6161.1958.tb00292.x","volume":"20","author":"DR Cox","year":"1958","unstructured":"Cox, D. R. The Regression Analysis of Binary Sequences. J. R. Stat. Soc. Ser. B Methodol. 20, 215\u2013242 (1958).","journal-title":"J. R. Stat. Soc. Ser. B Methodol."},{"key":"43525_CR29","doi-asserted-by":"crossref","unstructured":"Jia, Y. et al. Caffe: Convolutional Architecture for Fast Feature Embedding. ArXiv Prepr. ArXiv14085093 (2014).","DOI":"10.1145\/2647868.2654889"},{"key":"43525_CR30","first-page":"2825","volume":"12","author":"F Pedregosa","year":"2011","unstructured":"Pedregosa, F. et al. Scikit-learn: Machine Learning in Python. J. Mach. Learn. Res. 12, 2825\u20132830 (2011).","journal-title":"J. Mach. Learn. Res."},{"key":"43525_CR31","first-page":"604","volume":"43","author":"C Franklin","year":"2017","unstructured":"Franklin, C., Livingstone, E., Roesch, A., Schilling, B. & Schadendorf, D. Immunotherapy in melanoma: Recent advances and future directions. New Roads Melanoma Manag. 43, 604\u2013611 (2017).","journal-title":"New Roads Melanoma Manag."},{"key":"43525_CR32","doi-asserted-by":"publisher","first-page":"e1286436","DOI":"10.1080\/2162402X.2017.1286436","volume":"6","author":"A Berthel","year":"2017","unstructured":"Berthel, A. et al. Detailed resolution analysis reveals spatial T cell heterogeneity in the invasive margin of colorectal cancer liver metastases associated with improved survival. OncoImmunology 6, e1286436 (2017).","journal-title":"OncoImmunology"},{"key":"43525_CR33","doi-asserted-by":"publisher","first-page":"568","DOI":"10.1038\/nature13954","volume":"515","author":"PC Tumeh","year":"2014","unstructured":"Tumeh, P. C. et al. PD-1 blockade induces responses by inhibiting adaptive immune resistance. Nature 515, 568\u2013571 (2014).","journal-title":"Nature"}],"container-title":["Scientific Reports"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41598-019-43525-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-019-43525-8","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-019-43525-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,18]],"date-time":"2024-07-18T01:51:17Z","timestamp":1721267477000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41598-019-43525-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,5,15]]},"references-count":33,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2019,12]]}},"alternative-id":["43525"],"URL":"https:\/\/doi.org\/10.1038\/s41598-019-43525-8","relation":{},"ISSN":["2045-2322"],"issn-type":[{"value":"2045-2322","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,5,15]]},"assertion":[{"value":"4 December 2018","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 April 2019","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"15 May 2019","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"G.M., M.C. and G.B. declare no competing interests. N.H., R.S., K.N., A.M., N.B., C.V. and G.S. are full time employees of Definiens AG. P.A.A. has\/had consulting or advisory role for BMS, Roche-Genentech, MSD, Array, Novartis, Merck Serono, Pierre Fabre, Incyte, Genmab, Newlink Genetics, Medimmune, AstraZeneca, Syndax, Sun Pharma, Sanofi, Idera, Ultimovacs. He also received research funds from BMS, Roche-Genentech, Array, and travel support from MSD.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing Interests"}}],"article-number":"7449"}}