{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,6]],"date-time":"2025-11-06T12:25:18Z","timestamp":1762431918006,"version":"build-2065373602"},"reference-count":52,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2020,12,3]],"date-time":"2020-12-03T00:00:00Z","timestamp":1606953600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Imaging"],"abstract":"<jats:p>Immunotherapy is regarded as one of the most significant breakthroughs in cancer treatment. Unfortunately, only a small percentage of patients respond properly to the treatment. Moreover, to date, there are no efficient bio-markers able to early discriminate the patients eligible for this treatment. In order to help overcome these limitations, an innovative non-invasive deep pipeline, integrating Computed Tomography (CT) imaging, is investigated for the prediction of a response to immunotherapy treatment. We report preliminary results collected as part of a case study in which we validated the implemented method on a clinical dataset of patients affected by Metastatic Urothelial Carcinoma. The proposed pipeline aims to discriminate patients with high chances of response from those with disease progression. Specifically, the authors propose ad-hoc 3D Deep Networks integrating Self-Attention mechanisms in order to estimate the immunotherapy treatment response from CT-scan images and such hemato-chemical data of the patients. The performance evaluation (average accuracy close to 92%) confirms the effectiveness of the proposed approach as an immunotherapy treatment response biomarker.<\/jats:p>","DOI":"10.3390\/jimaging6120133","type":"journal-article","created":{"date-parts":[[2020,12,3]],"date-time":"2020-12-03T11:15:43Z","timestamp":1606994143000},"page":"133","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":14,"title":["3D Non-Local Neural Network: A Non-Invasive Biomarker for Immunotherapy Treatment Outcome Prediction. Case-Study: Metastatic Urothelial Carcinoma"],"prefix":"10.3390","volume":"6","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1766-3065","authenticated-orcid":false,"given":"Francesco","family":"Rundo","sequence":"first","affiliation":[{"name":"STMicroelectronics\u2014ADG Central R&amp;D Division, 95125 Catania, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Giuseppe Luigi","family":"Banna","sequence":"additional","affiliation":[{"name":"Medical Oncology Department, United Lincolnshire NHS Hospital Trust, Lincoln LN2, Lincolnshire, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Luca","family":"Prezzavento","sequence":"additional","affiliation":[{"name":"DIEEI, University of Catania, 95125 Catania, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Francesca","family":"Trenta","sequence":"additional","affiliation":[{"name":"IPLAB, University of Catania, 95125 Catania, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sabrina","family":"Conoci","sequence":"additional","affiliation":[{"name":"Department of Chemical, Biological, Pharmaceutical and Environmental Sciences, University of Messina, 98100 Messina, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6127-2470","authenticated-orcid":false,"given":"Sebastiano","family":"Battiato","sequence":"additional","affiliation":[{"name":"IPLAB, University of Catania, 95125 Catania, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,12,3]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"E359","DOI":"10.1002\/ijc.29210","article-title":"Cancer incidence and mortality worldwide: Sources, methods and major patterns in Globocan 2012","volume":"136","author":"Ferlay","year":"2015","journal-title":"Int. J. Cancer"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1097\/00005392-200101000-00012","article-title":"De Novo Muscle Invasive Bladder Cancer: Is There a Change in Trend?","volume":"165","author":"Vaidya","year":"2001","journal-title":"J. Urol."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1200\/JCO.2011.37.3571","article-title":"Randomized Phase II\/III Trial Assessing Gemcitabine\/Carboplatin and Methotrexate\/Carboplatin\/Vinblastine in Patients With Advanced Urothelial Cancer Who Are Unfit for Cisplatin-Based Chemotherapy: EORTC Study 30986","volume":"30","author":"Bellmunt","year":"2012","journal-title":"J. Clin. Oncol."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"341","DOI":"10.1016\/j.ctrv.2015.03.004","article-title":"Molecular biology and targeted therapies for urothelial carcinoma","volume":"41","author":"Seront","year":"2015","journal-title":"Cancer Treat. Rev."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"748","DOI":"10.1016\/S0140-6736(17)33297-X","article-title":"Atezolizumab versus chemotherapy in patients with platinum-treated locally advanced or metastatic urothelial carcinoma (IMvigor211): A multicentre, open-label, phase 3 randomised controlled trial","volume":"391","author":"Powles","year":"2018","journal-title":"Lancet"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1015","DOI":"10.1056\/NEJMoa1613683","article-title":"Pembrolizumab as Second-Line Therapy for Advanced Urothelial Carcinoma","volume":"376","author":"Bellmunt","year":"2017","journal-title":"N. Engl. J. Med."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1016\/j.eururo.2019.03.015","article-title":"Primary Results from SAUL, a Multinational Single-arm Safety Study of Atezolizumab Therapy for Locally Advanced or Metastatic Urothelial or Nonurothelial Carcinoma of the Urinary Tract","volume":"76","author":"Sternberg","year":"2019","journal-title":"Eur. Urol."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1016\/S0140-6736(16)32455-2","article-title":"Atezolizumab as first-line treatment in cisplatin-ineligible patients with locally advanced and metastatic urothelial carcinoma: A single-arm, multicentre, phase 2 trial","volume":"389","author":"Balar","year":"2017","journal-title":"Lancet"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"4523","DOI":"10.1200\/JCO.2018.36.15_suppl.4523","article-title":"Atezolizumab (atezo) in first-line cisplatin-ineligible or platinum-treated locally advanced or metastatic urothelial cancer (mUC): Long-term efficacy from phase 2 study IMvigor210","volume":"36","author":"Balar","year":"2018","journal-title":"J. Clin. Oncol."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1483","DOI":"10.1016\/S1470-2045(17)30616-2","article-title":"First-line pembrolizumab in cisplatin-ineligible patients with locally advanced and unresectable or metastatic urothelial cancer (Keynote-052): A multicentre, single-arm, phase 2 study","volume":"18","author":"Balar","year":"2017","journal-title":"Lancet Oncol."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"4524","DOI":"10.1200\/JCO.2018.36.15_suppl.4524","article-title":"(Joaquim); Powles, T.; Bajorin, D.F.; Hahn, N.M.; De Wit, R.; et al. Updated efficacy and safety of KEYNOTE-052: A single-arm phase 2 study investigating first-line pembrolizumab (pembro) in cisplatin-ineligible advanced urothelial cancer (UC)","volume":"36","author":"Vuky","year":"2018","journal-title":"J. Clin. Oncol."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"4501","DOI":"10.1200\/JCO.2016.34.15_suppl.4501","article-title":"Efficacy and safety of nivolumab monotherapy in metastatic urothelial cancer (mUC): Results from the phase I\/II CheckMate 032 study","volume":"34","author":"Seliger","year":"2016","journal-title":"J. Clin. Oncol."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"3119","DOI":"10.1200\/JCO.2016.67.9761","article-title":"Safety and Efficacy of Durvalumab (MEDI4736), an Anti\u2013Programmed Cell Death Ligand-1 Immune Checkpoint Inhibitor, in Patients With Advanced Urothelial Bladder Cancer","volume":"34","author":"Massard","year":"2016","journal-title":"J. Clin. Oncol."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"2117","DOI":"10.1200\/JCO.2016.71.6795","article-title":"Avelumab, an Anti\u2013Programmed Death-Ligand 1 Antibody, In Patients With Refractory Metastatic Urothelial Carcinoma: Results From a Multicenter, Phase Ib Study","volume":"35","author":"Apolo","year":"2017","journal-title":"J. Clin. Oncol."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"94","DOI":"10.1186\/s40425-017-0299-1","article-title":"Biomarkers for immunotherapy in bladder cancer: A moving target","volume":"5","author":"Aggen","year":"2017","journal-title":"J. Immunother. Cancer"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"79","DOI":"10.3233\/CBM-2011-0232","article-title":"CXCR4 and CXCL12 immunoreactivities differentiate primary non-small-cell lung cancer with or without brain metastases","volume":"10","author":"Paratore","year":"2012","journal-title":"Cancer Biomark."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"957","DOI":"10.1049\/iet-cvi.2018.5195","article-title":"Evaluation of Levenberg\u2013Marquardt neural networks and stacked autoencoders clustering for skin lesion analysis, screening and follow-up","volume":"12","author":"Rundo","year":"2018","journal-title":"IET Comput. Vis."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"934","DOI":"10.1001\/jamaoncol.2019.0626","article-title":"Tumor Mutation Burden\u2014From Hopes to Doubts","volume":"5","author":"Addeo","year":"2019","journal-title":"JAMA Oncol."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"172","DOI":"10.3389\/fmed.2019.00172","article-title":"The Promise of Digital Biopsy for the Prediction of Tumor Molecular Features and Clinical Outcomes Associated with Immunotherapy","volume":"6","author":"Banna","year":"2019","journal-title":"Front. Med."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"441","DOI":"10.1016\/j.ejca.2011.11.036","article-title":"Radiomics: Extracting more information from medical images using advanced feature analysis","volume":"48","author":"Lambin","year":"2012","journal-title":"Eur. J. Cancer"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Wang, X., Girshick, R., Gupta, A., and He, K. (2018, January 18\u201323). Non-local Neural Networks. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00813"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Williams, H., Cattani, L., Li, W., Tabassian, M., Vercauteren, T., Deprest, J., and D\u2019Hooge, J. (2019, January 6\u20139). 3D Convolutional Neural Network for Segmentation of the Urethra in Volumetric Ultrasound of the Pelvic Floor. Proceedings of the 2019 IEEE International Ultrasonics Symposium (IUS), Glasgow, UK.","DOI":"10.1109\/ULTSYM.2019.8925792"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Moradi, P., and Jamzad, M. (2019, January 6\u20137). Detecting Lung Cancer Lesions in CT Images using 3D Convolutional Neural Networks. Proceedings of the 2019 4th International Conference on Pattern Recognition and Image Analysis (IPRIA), Tehran, Iran.","DOI":"10.1109\/PRIA.2019.8785971"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"124","DOI":"10.1016\/j.compbiomed.2015.05.015","article-title":"Prediction of mortality after radical cystectomy for bladder cancer by machine learning techniques","volume":"63","author":"Wang","year":"2015","journal-title":"Comput. Biol. Med."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"5814","DOI":"10.1002\/mp.12510","article-title":"Urinary bladder cancer staging in CT urography using machine learning","volume":"44","author":"Garapati","year":"2017","journal-title":"Med. Phys."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Hasnain, Z., Mason, J., Gill, K., Miranda, G., Gill, I.S., Kuhn, P., and Newton, P.K. (2019). Machine learning models for predicting post-cystectomy recurrence and survival in bladder cancer patients. PLoS ONE, 14.","DOI":"10.1371\/journal.pone.0210976"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1882","DOI":"10.1118\/1.4944498","article-title":"Urinary bladder segmentation in CT urography using deep-learning convolutional neural network and level sets","volume":"43","author":"Cha","year":"2016","journal-title":"Med. Phys."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Gordon, M., Hadjiiski, L., Cha, K., Chan, H.-P., Samala, R., Cohan, R.H., and Caoili, E.M. (2017). Segmentation of inner and outer bladder wall using deep-learning convolutional neural network in CT urography. Medical Imaging 2017: Computer-Aided Diagnosis, Proceedings of the SPIE Medical Imaging, Orlando, FL, USA, 13\u201316 February 2017, International Society for Optics and Photonics.","DOI":"10.1117\/12.2255528"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Ma, X., Hadjiiski, L.M., Wei, J., Chan, H.-P., Cohan, R.H., Caoili, E.M., Samala, R.K., Zhou, C., Lu, Y., and Cha, K.H. (2019). 2D and 3D bladder segmentation using U-Net-based deep-learning. Medical Imaging 2019: Computer-Aided Diagnosis, Proceedings of the SPIE Medical Imaging, San Diego, CA, USA, 13 March 2019, International Society for Optics and Photonics.","DOI":"10.1117\/12.2511890"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"714","DOI":"10.1016\/j.eururo.2019.08.032","article-title":"Augmented Bladder Tumor Detection Using Deep Learning","volume":"76","author":"Shkolyar","year":"2019","journal-title":"Eur. Urol."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"201","DOI":"10.18383\/j.tom.2018.00036","article-title":"Deep Learning Approach for Assessment of Bladder Cancer Treatment Response","volume":"5","author":"Wu","year":"2019","journal-title":"Tomography"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1145\/3065386","article-title":"Imagenet classification with deep convolutional neural networks","volume":"60","author":"Krizhevsky","year":"2017","journal-title":"Commun. ACM"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41598-017-09315-w","article-title":"Bladder Cancer Treatment Response Assessment in CT using Radiomics with Deep-Learning","volume":"7","author":"Cha","year":"2017","journal-title":"Sci. Rep."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Rundo, F., Spampinato, C., Banna, G., and Conoci, S. (2019). Advanced Deep Learning Embedded Motion Radiomics Pipeline for Predicting Anti-PD-1\/PD-L1 Immunotherapy Response in the Treatment of Bladder Cancer: Preliminary Results. Electronics, 8.","DOI":"10.3390\/electronics8101134"},{"key":"ref_35","unstructured":"Holzinger, A., Biemann, C., Kell, D., and Pattichis, C.S. (2017). What do we need to build explainable AI systems for the medical domain?. arXiv."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Samek, W., Montavon, G., Vedaldi, A., Hansen, L.K., and M\u00fcller, K.R. (2019). Explainable AI: Interpreting, Explaining and Visualizing Deep Learning, Springer.","DOI":"10.1007\/978-3-030-28954-6"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1038\/s41551-018-0324-9","article-title":"An explainable deep-learning algorithm for the detection of acute intracranial haemorrhage from small datasets","volume":"3","author":"Lee","year":"2018","journal-title":"Nat. Biomed. Eng."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1016\/j.inffus.2019.12.012","article-title":"Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI","volume":"58","author":"Arrieta","year":"2020","journal-title":"Inf. Fusion"},{"key":"ref_39","first-page":"1","article-title":"What clinicians want: Contextualizing explainable machine learning for clinical end use","volume":"106","author":"Tonekaboni","year":"2019","journal-title":"Mach. Learn. Res."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D. (2017, January 22\u201329). Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.74"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"228","DOI":"10.1016\/j.ejca.2008.10.026","article-title":"New response evaluation criteria in solid tumours: Revised RECIST guideline (version 1.1)","volume":"45","author":"Eisenhauer","year":"2009","journal-title":"Eur. J. Cancer"},{"key":"ref_42","unstructured":"Jordan, M.I., LeCun, Y., and Solla, S.A. (2017). Attention is all you need. Advances in Neural Information Processing System, Proceedings of the First 12 Conferences (The MIT Press), 21 November 2001, The MIT Press."},{"key":"ref_43","unstructured":"Huang, G., Liu, Z., Pleiss, G., Van Der Maaten, L., and Weinberger, K. (2019). Convolutional Networks with Dense Connectivity. IEEE Trans. Pattern Anal. Mach. Intell., 1."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Chen, H., and Cao, P. (2019, January 12\u201314). Deep Learning Based Data Augmentation and Classification for Limited Medical Data Learning. Proceedings of the 2019 IEEE International Conference on Power, Intelligent Computing and Systems; Institute of Electrical and Electronics Engineers (IEEE), Shenyang, China.","DOI":"10.1109\/ICPICS47731.2019.8942411"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s41747-019-0134-1","article-title":"Imaging of tumour response to immunotherapy","volume":"4","author":"Dromain","year":"2020","journal-title":"Eur. Radiol. Exp."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"561","DOI":"10.3389\/fphar.2017.00561","article-title":"PD-1 and PD-L1 Checkpoint Signaling Inhibition for Cancer Immunotherapy: Mechanism, Combinations, and Clinical Outcome","volume":"8","author":"Alsaab","year":"2017","journal-title":"Front. Pharmacol."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"4171","DOI":"10.2147\/CMAR.S176937","article-title":"Clinicopathological and prognostic value of PD-L1 in urothelial carcinoma: A meta-analysis","volume":"11","author":"Ding","year":"2019","journal-title":"Cancer Manag. Res."},{"key":"ref_48","first-page":"14","article-title":"A review of the PD-1\/PD-L1 checkpoint in bladder cancer: From mediator of immune escape to target for treatment 1 1MPS is an investor in and consultant for Urogen. SAP is consultant and advisor for Vaccinex. The remaining authors have nothing to disclose","volume":"35","author":"Zhou","year":"2017","journal-title":"Urol. Oncol. Semin. Orig. Investig."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"e493","DOI":"10.1200\/EDBK_160766","article-title":"Biomarkers for Immunotherapy: Current Developments and Challenges","volume":"36","author":"Spencer","year":"2016","journal-title":"Am. Soc. Clin. Oncol. Educ. Book"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Rundo, F., Trenta, F., Di Stallo, A.L., and Battiato, S. (2019). Advanced Markov-Based Machine Learning Framework for Making Adaptive Trading System. Computation, 7.","DOI":"10.3390\/computation7010004"},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Rundo, F., Trenta, F., Di Stallo, A.L., and Battiato, S. (2019). Grid Trading System Robot (GTSbot): A Novel Mathematical Algorithm for trading FX Market. Appl. Sci., 9.","DOI":"10.3390\/app9091796"},{"key":"ref_52","unstructured":"Rundo, F., Petralia, S., Fallica, G., and Libertino, S. (2019). Lecture Notes in Electrical Engineering, Springer."}],"container-title":["Journal of Imaging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2313-433X\/6\/12\/133\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:41:09Z","timestamp":1760179269000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2313-433X\/6\/12\/133"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,12,3]]},"references-count":52,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2020,12]]}},"alternative-id":["jimaging6120133"],"URL":"https:\/\/doi.org\/10.3390\/jimaging6120133","relation":{},"ISSN":["2313-433X"],"issn-type":[{"type":"electronic","value":"2313-433X"}],"subject":[],"published":{"date-parts":[[2020,12,3]]}}}