{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,26]],"date-time":"2026-06-26T03:51:43Z","timestamp":1782445903337,"version":"3.54.5"},"reference-count":33,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2022,5,9]],"date-time":"2022-05-09T00:00:00Z","timestamp":1652054400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Imaging"],"abstract":"<jats:p>It is proven that radiomic characteristics extracted from the tumor region are predictive. The first step in radiomic analysis is the segmentation of the lesion. However, this task is time consuming and requires a highly trained physician. This process could be automated using computer-aided detection (CAD) tools. Current state-of-the-art methods are trained in a supervised learning setting, which requires a lot of data that are usually not available in the medical imaging field. The challenge is to train one model to segment different types of tumors with only a weak segmentation ground truth. In this work, we propose a prediction framework including a 3D tumor segmentation in positron emission tomography (PET) images, based on a weakly supervised deep learning method, and an outcome prediction based on a 3D-CNN classifier applied to the segmented tumor regions. The key step is to locate the tumor in 3D. We propose to (1) calculate two maximum intensity projection (MIP) images from 3D PET images in two directions, (2) classify the MIP images into different types of cancers, (3) generate the class activation maps through a multitask learning approach with a weak prior knowledge, and (4) segment the 3D tumor region from the two 2D activation maps with a proposed new loss function for the multitask. The proposed approach achieves state-of-the-art prediction results with a small data set and with a weak segmentation ground truth. Our model was tested and validated for treatment response and survival in lung and esophageal cancers on 195 patients, with an area under the receiver operating characteristic curve (AUC) of 67% and 59%, respectively, and a dice coefficient of 73% and 0.77% for tumor segmentation.<\/jats:p>","DOI":"10.3390\/jimaging8050130","type":"journal-article","created":{"date-parts":[[2022,5,9]],"date-time":"2022-05-09T21:23:27Z","timestamp":1652131407000},"page":"130","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Weakly Supervised Tumor Detection in PET Using Class Response for Treatment Outcome Prediction"],"prefix":"10.3390","volume":"8","author":[{"given":"Amine","family":"Amyar","sequence":"first","affiliation":[{"name":"General Electric Healthcare, 78530 Buc, France"},{"name":"LITIS-EA4108-Quantif, University of Rouen, 76800 Rouen, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Romain","family":"Modzelewski","sequence":"additional","affiliation":[{"name":"LITIS-EA4108-Quantif, University of Rouen, 76800 Rouen, France"},{"name":"Nuclear Medicine Department, Henri Becquerel Center, 76038 Rouen, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pierre","family":"Vera","sequence":"additional","affiliation":[{"name":"LITIS-EA4108-Quantif, University of Rouen, 76800 Rouen, France"},{"name":"Nuclear Medicine Department, Henri Becquerel Center, 76038 Rouen, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6982-8645","authenticated-orcid":false,"given":"Vincent","family":"Morard","sequence":"additional","affiliation":[{"name":"General Electric Healthcare, 78530 Buc, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8785-6917","authenticated-orcid":false,"given":"Su","family":"Ruan","sequence":"additional","affiliation":[{"name":"LITIS-EA4108-Quantif, University of Rouen, 76800 Rouen, France"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,5,9]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"563","DOI":"10.1148\/radiol.2015151169","article-title":"Radiomics: Images are more than pictures, they are data","volume":"278","author":"Gillies","year":"2016","journal-title":"Radiology"},{"key":"ref_2","first-page":"324","article-title":"Radiomics-net: Convolutional neural networks on FDG PET images for predicting cancer treatment response","volume":"59","author":"Amyar","year":"2018","journal-title":"J. Nucl. Med."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"257","DOI":"10.1016\/j.media.2016.05.007","article-title":"Selecting radiomic features from FDG-PET images for cancer treatment outcome prediction","volume":"32","author":"Lian","year":"2016","journal-title":"Med. Image Anal."},{"key":"ref_4","first-page":"1097","article-title":"Imagenet classification with deep convolutional neural networks","volume":"25","author":"Krizhevsky","year":"2012","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014, January 23\u201328). Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.81"},{"key":"ref_6","unstructured":"Rajpurkar, P., Irvin, J., Zhu, K., Yang, B., Mehta, H., Duan, T., Ding, D., Bagul, A., Langlotz, C., and Shpanskaya, K. (2017). Chexnet: Radiologist-level pneumonia detection on chest X-rays with deep learning. arXiv."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1038\/s41591-018-0268-3","article-title":"Cardiologist-level arrhythmia detection and classification in ambulatory electrocardiograms using a deep neural network","volume":"25","author":"Hannun","year":"2019","journal-title":"Nat. Med."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1016\/j.cpet.2021.09.010","article-title":"AI-Based Detection, Classification and Prediction\/Prognosis in Medical Imaging: Towards Radiophenomics","volume":"17","author":"Yousefirizi","year":"2022","journal-title":"PET Clin."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Mahendran, A., and Vedaldi, A. (2015, January 7\u201312). Understanding deep image representations by inverting them. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7299155"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Zeiler, M.D., and Fergus, R. (2014). Visualizing and understanding convolutional networks. European Conference on Computer Vision, Springer.","DOI":"10.1007\/978-3-319-10590-1_53"},{"key":"ref_11","unstructured":"Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., and Torralba, A. (2014). Object detectors emerge in deep scene cnns. arXiv."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., and Torralba, A. (2016, January 27\u201330). Learning deep features for discriminative localization. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.319"},{"key":"ref_13","unstructured":"Lin, M., Chen, Q., and Yan, S. (2013). Network in network. arXiv."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7\u201312). Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D. (2017, January 22\u201329). Grad-cam: Visual explanations from deep networks via gradient-based localization. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.74"},{"key":"ref_16","first-page":"1212","article-title":"Contribution of class activation map on WB PET deep features for primary tumour classification","volume":"60","author":"Amyar","year":"2019","journal-title":"J. Nucl. Med."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"He, K., Gkioxari, G., Doll\u00e1r, P., and Girshick, R. (2017, January 22\u201329). Mask r-cnn. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.322"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Bazzani, L., Bergamo, A., Anguelov, D., and Torresani, L. (2016, January 7\u201310). Self-taught object localization with deep networks. Proceedings of the 2016 IEEE Winter Conference on Applications of Computer Vision (WACV), Lake Placid, NY, USA.","DOI":"10.1109\/WACV.2016.7477688"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1109\/TPAMI.2016.2535231","article-title":"Weakly supervised object localization with multi-fold multiple instance learning","volume":"39","author":"Cinbis","year":"2016","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Ahn, J., Cho, S., and Kwak, S. (2019, January 15\u201320). Weakly supervised learning of instance segmentation with inter-pixel relations. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00231"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Zhou, Y., Zhu, Y., Ye, Q., Qiu, Q., and Jiao, J. (2018, January 18\u201323). Weakly supervised instance segmentation using class peak response. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00399"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"42","DOI":"10.1016\/j.compmedimag.2016.12.002","article-title":"Feature selection for outcome prediction in oesophageal cancer using genetic algorithm and random forest classifier","volume":"60","author":"Paul","year":"2017","journal-title":"Comput. Med. Imaging Graph."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"13206","DOI":"10.1038\/s41598-017-13448-3","article-title":"A comparative study of machine learning methods for time-to-event survival data for radiomics risk modelling","volume":"7","author":"Leger","year":"2017","journal-title":"Sci. Rep."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1145","DOI":"10.1109\/TBME.2015.2485779","article-title":"MAPS: A quantitative radiomics approach for prostate cancer detection","volume":"63","author":"Cameron","year":"2015","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1007\/s00259-016-3427-0","article-title":"Characterization of PET\/CT images using texture analysis: The past, the present\u2026 any future?","volume":"44","author":"Hatt","year":"2017","journal-title":"Eur. J. Nucl. Med. Mol. Imaging"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1935","DOI":"10.1109\/TBME.2018.2844188","article-title":"A radiomics approach with CNN for shear-wave elastography breast tumor classification","volume":"65","author":"Zhou","year":"2018","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Hosny, A., Parmar, C., Coroller, T.P., Grossmann, P., Zeleznik, R., Kumar, A., Bussink, J., Gillies, R.J., Mak, R.H., and Aerts, H.J. (2018). Deep learning for lung cancer prognostication: A retrospective multi-cohort radiomics study. PLoS Med., 15.","DOI":"10.1371\/journal.pmed.1002711"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"225","DOI":"10.1109\/TRPMS.2019.2896399","article-title":"3-d rpet-net: Development of a 3-d pet imaging convolutional neural network for radiomics analysis and outcome prediction","volume":"3","author":"Amyar","year":"2019","journal-title":"IEEE Trans. Radiat. Plasma Med. Sci."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"433","DOI":"10.1148\/radiographics.17.2.9084083","article-title":"Use of maximum intensity projections in CT angiography: A basic review","volume":"17","author":"Prokop","year":"1997","journal-title":"Radiographics"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"325","DOI":"10.1007\/s00330-005-2871-1","article-title":"Value of axial and coronal maximum intensity projection (MIP) images in the detection of pulmonary nodules by multislice spiral CT: Comparison with axial 1-mm and 5-mm slices","volume":"16","author":"Valencia","year":"2006","journal-title":"Eur. Radiol."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"2481","DOI":"10.1109\/TPAMI.2016.2644615","article-title":"Segnet: A deep convolutional encoder-decoder architecture for image segmentation","volume":"39","author":"Badrinarayanan","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"94","DOI":"10.1016\/j.isprsjprs.2020.01.013","article-title":"ResUNet-a: A deep learning framework for semantic segmentation of remotely sensed data","volume":"162","author":"Diakogiannis","year":"2020","journal-title":"ISPRS J. Photogramm. Remote Sens."}],"container-title":["Journal of Imaging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2313-433X\/8\/5\/130\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:08:10Z","timestamp":1760137690000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2313-433X\/8\/5\/130"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,5,9]]},"references-count":33,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2022,5]]}},"alternative-id":["jimaging8050130"],"URL":"https:\/\/doi.org\/10.3390\/jimaging8050130","relation":{},"ISSN":["2313-433X"],"issn-type":[{"value":"2313-433X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,5,9]]}}}