{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,5]],"date-time":"2026-03-05T15:38:22Z","timestamp":1772725102336,"version":"3.50.1"},"reference-count":46,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2022,4,26]],"date-time":"2022-04-26T00:00:00Z","timestamp":1650931200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Oil spillage over a sea or ocean surface is a threat to marine and coastal ecosystems. Spaceborne synthetic aperture radar (SAR) data have been used efficiently for the detection of oil spills due to their operational capability in all-day all-weather conditions. The problem is often modeled as a semantic segmentation task. The images need to be segmented into multiple regions of interest such as sea surface, oil spill, lookalikes, ships, and land. Training of a classifier for this task is particularly challenging since there is an inherent class imbalance. In this work, we train a convolutional neural network (CNN) with multiple feature extractors for pixel-wise classification and introduce a new loss function, namely, \u201cgradient profile\u201d (GP) loss, which is in fact the constituent of the more generic spatial profile loss proposed for image translation problems. For the purpose of training, testing, and performance evaluation, we use a publicly available dataset with selected oil spill events verified by the European Maritime Safety Agency (EMSA). The results obtained show that the proposed CNN trained with a combination of GP, Jaccard, and focal loss functions can detect oil spills with an intersection over union (IoU) value of 63.95%. The IoU value for sea surface, lookalikes, ships, and land class is 96.00%, 60.87%, 74.61%, and 96.80%, respectively. The mean intersection over union (mIoU) value for all the classes is 78.45%, which accounts for a 13% improvement over the state of the art for this dataset. Moreover, we provide extensive ablation on different convolutional neural networks (CNNs) and vision transformers (ViTs)-based hybrid models to demonstrate the effectiveness of adding GP loss as an additional loss function for training. Results show that GP loss significantly improves the mIoU and F1 scores for CNNs as well as ViTs-based hybrid models. GP loss turns out to be a promising loss function in the context of deep learning with SAR images.<\/jats:p>","DOI":"10.3390\/rs14092085","type":"journal-article","created":{"date-parts":[[2022,4,26]],"date-time":"2022-04-26T21:37:53Z","timestamp":1651009073000},"page":"2085","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":27,"title":["Comparison of CNNs and Vision Transformers-Based Hybrid Models Using Gradient Profile Loss for Classification of Oil Spills in SAR Images"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0092-6853","authenticated-orcid":false,"given":"Abdul","family":"Basit","sequence":"first","affiliation":[{"name":"Remote Sensing and Spatial Analytics Lab, Information Technology University of the Punjab (ITU), Lahore 54000, Pakistan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1695-7694","authenticated-orcid":false,"given":"Muhammad Adnan","family":"Siddique","sequence":"additional","affiliation":[{"name":"Remote Sensing and Spatial Analytics Lab, Information Technology University of the Punjab (ITU), Lahore 54000, Pakistan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1974-8268","authenticated-orcid":false,"given":"Muhammad Khurram","family":"Bhatti","sequence":"additional","affiliation":[{"name":"Remote Sensing and Spatial Analytics Lab, Information Technology University of the Punjab (ITU), Lahore 54000, Pakistan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1271-0005","authenticated-orcid":false,"given":"Muhammad Saquib","family":"Sarfraz","sequence":"additional","affiliation":[{"name":"Institute for Anthropomatics and Robotics, Karlsruhe Institute of Technology (KIT), 76131 Karlsruhe, Germany"}]}],"member":"1968","published-online":{"date-parts":[[2022,4,26]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"2931","DOI":"10.1109\/JPROC.2012.2196250","article-title":"Remote Sensing of Ocean Oil-Spill Pollution","volume":"100","author":"Solberg","year":"2012","journal-title":"Proc. IEEE"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1916","DOI":"10.1109\/36.774704","article-title":"Automatic detection of oil spills in ERS SAR images","volume":"37","author":"Solberg","year":"1999","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1016\/j.marpolbul.2014.03.059","article-title":"Review of oil spill remote sensing","volume":"83","author":"Fingas","year":"2014","journal-title":"Mar. Pollut. Bull."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"22","DOI":"10.1109\/MGRS.2016.2540798","article-title":"Deep Learning for Remote Sensing Data: A Technical Tutorial on the State of the Art","volume":"4","author":"Zhang","year":"2016","journal-title":"IEEE Geosci. Remote Sens. Mag."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Chen, Y., Li, Y., and Wang, J. (2020). An End-to-End Oil-Spill Monitoring Method for Multisensory Satellite Images Based on Deep Semantic Segmentation. Sensors, 20.","DOI":"10.3390\/s20030725"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Liu, Y., Wang, L., Zhao, L., and Yu, Z. (2020). Advances in Natural Computation, Fuzzy Systems and Knowledge Discovery, Springer International Publishing.","DOI":"10.1007\/978-3-030-32456-8"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation. Medical Image Computing and Computer-Assisted Intervention\u2014MICCAI 2015, Springer International Publishing.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Ghosh, A., Ehrlich, M., Shah, S., Davis, L.S., and Chellappa, R. (2018, January 18\u201322). Stacked U-Nets for Ground Material Segmentation in Remote Sensing Imagery. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPRW.2018.00047"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"3954","DOI":"10.1109\/JSTARS.2018.2833382","article-title":"DeepUNet: A Deep Fully Convolutional Network for Pixel-Level Sea-Land Segmentation","volume":"11","author":"Li","year":"2018","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"75","DOI":"10.1109\/JSTARS.2020.3036914","article-title":"Snow Avalanche Segmentation in SAR Images With Fully Convolutional Neural Networks","volume":"14","author":"Bianchi","year":"2021","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"834","DOI":"10.1109\/TPAMI.2017.2699184","article-title":"DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs","volume":"40","author":"Chen","year":"2018","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_12","unstructured":"Tan, M., and Le, Q.V. (2021, January 18\u201324). EfficientNetV2: Smaller Models and Faster Training. Proceedings of the 2021 International Conference on Machine Learning, Virtual."},{"key":"ref_13","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L.U., and Polosukhin, I. (2017, January 4\u20139). Attention is All you Need. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_14","unstructured":"Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2018). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv."},{"key":"ref_15","first-page":"1877","article-title":"Language Models are Few-Shot Learners","volume":"33","author":"Brown","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Wang, X., Girshick, R., Gupta, A., and He, K. (2018, January 18\u201322). Non-local Neural Networks. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00813"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Bello, I., Zoph, B., Le, Q., Vaswani, A., and Shlens, J. (November, January 27). Attention Augmented Convolutional Networks. Proceedings of the 2019 IEEE\/CVF International Conference on Computer Vision (ICCV), Seol, Korea.","DOI":"10.1109\/ICCV.2019.00338"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Zhuoran, S., Mingyuan, Z., Haiyu, Z., Shuai, Y., and Hongsheng, L. (2021, January 5\u20139). Efficient Attention: Attention with Linear Complexities. Proceedings of the 2021 IEEE Winter Conference on Applications of Computer Vision (WACV), Virtual.","DOI":"10.1109\/WACV48630.2021.00357"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Guo, J., Han, K., Wu, H., Xu, C., Tang, Y., Xu, C., and Wang, Y. (2021). CMT: Convolutional Neural Networks Meet Vision Transformers. arXiv.","DOI":"10.1109\/CVPR52688.2022.01186"},{"key":"ref_20","unstructured":"Dai, Z., Liu, H., Le, Q.V., and Tan, M. (2021). CoAtNet: Marrying Convolution and Attention for All Data Sizes. arXiv."},{"key":"ref_21","unstructured":"Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, MIT Press. Available online: http:\/\/www.deeplearningbook.org."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"264","DOI":"10.1016\/j.isprsjprs.2007.05.003","article-title":"Detection and discrimination between oil spills and look-alike phenomena through neural networks","volume":"62","author":"Topouzelis","year":"2007","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"2355","DOI":"10.1109\/JSTARS.2013.2251864","article-title":"Satellite Oil Spill Detection Using Artificial Neural Networks","volume":"6","author":"Singha","year":"2013","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"2517","DOI":"10.1109\/JSTARS.2013.2244061","article-title":"Oil Spill Mapping and Measurement in the Gulf of Mexico With Textural Classifier Neural Network Algorithm (TCNNA)","volume":"6","author":"MacDonald","year":"2013","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Guo, H., Wei, G., and An, J. (2018). Dark Spot Detection in SAR Images of Oil Spill Using Segnet. Appl. Sci., 8.","DOI":"10.3390\/app8122670"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"2481","DOI":"10.1109\/TPAMI.2016.2644615","article-title":"SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation","volume":"39","author":"Badrinarayanan","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Orfanidis, G., Ioannidis, K., Avgerinakis, K., Vrochidis, S., and Kompatsiaris, I. (2018, January 7\u201310). A Deep Neural Network for Oil Spill Semantic Segmentation in SAR Images. Proceedings of the 2018 25th IEEE International Conference on Image Processing (ICIP), Athens, Greece.","DOI":"10.1109\/ICIP.2018.8451113"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Krestenitis, M., Orfanidis, G., Ioannidis, K., Avgerinakis, K., Vrochidis, S., and Kompatsiaris, I. (2019). Oil Spill Identification from Satellite Images Using Deep Neural Networks. Remote Sens., 11.","DOI":"10.3390\/rs11151762"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"190","DOI":"10.1016\/j.isprsjprs.2020.07.011","article-title":"A novel deep learning instance segmentation model for automated marine oil spill detection","volume":"167","author":"Yekeen","year":"2020","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Krestenitis, M., Orfanidis, G., Ioannidis, K., Avgerinakis, K., Vrochidis, S., and Kompatsiaris, I. (2018). Early Identification of Oil Spills in Satellite Images Using Deep CNNs. MultiMedia Modeling, Springer International Publishing.","DOI":"10.1007\/978-3-030-05710-7_35"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Shaban, M., Salim, R., Khalifeh, H.A., Khelifi, A., Shalaby, A., El-Mashad, S., Mahmoud, A., Ghazal, M., and El-Baz, A. (2021). A Deep-Learning Framework for the Detection of Oil Spills from SAR Data. Sensors, 21.","DOI":"10.3390\/s21072351"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Fan, Y., Rui, X., Zhang, G., Yu, T., Xu, X., and Poslad, S. (2021). Feature Merged Network for Oil Spill Detection Using SAR Images. Remote Sens., 13.","DOI":"10.3390\/rs13163174"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Basit, A., Siddique, M.A., and Sarfraz, M.S. (2021, January 11\u201316). Deep Learning Based Oil Spill Classification Using Unet Convolutional Neural Network. Proceedings of the 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS, Brussels, Belgium.","DOI":"10.1109\/IGARSS47720.2021.9553646"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"5217","DOI":"10.1109\/TGRS.2018.2812619","article-title":"Two-Stage Convolutional Neural Network for Ship and Spill Detection Using SLAR Images","volume":"56","author":"Gallego","year":"2018","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Zeng, K., and Wang, Y. (2020). A Deep Convolutional Neural Network for Oil Spill Detection from Spaceborne SAR Images. Remote Sens., 12.","DOI":"10.3390\/rs12061015"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"101851","DOI":"10.1016\/j.media.2020.101851","article-title":"Boundary loss for highly unbalanced segmentation","volume":"67","author":"Kervadec","year":"2021","journal-title":"Med. Image Anal."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"640","DOI":"10.1109\/TPAMI.2016.2572683","article-title":"Fully Convolutional Networks for Semantic Segmentation","volume":"39","author":"Shelhamer","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_38","unstructured":"Sarfraz, M.S., Seibold, C., Khalid, H., and Stiefelhagen, R. (2019). Content and Colour Distillation for Learning Image Translations with the Spatial Profile Loss. arXiv."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1016\/j.isprsjprs.2016.04.006","article-title":"Object-oriented approach to oil spill detection using ENVISAT ASAR images","volume":"118","author":"Konik","year":"2016","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1016\/j.isprsjprs.2012.01.005","article-title":"Oil spill feature selection and classification using decision tree forest on SAR image data","volume":"68","author":"Topouzelis","year":"2012","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"664","DOI":"10.1109\/TPAMI.2016.2598339","article-title":"Deep Visual-Semantic Alignments for Generating Image Descriptions","volume":"39","author":"Karpathy","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Vinyals, O., Toshev, A., Bengio, S., and Erhan, D. (2015, January 7\u201312). Show and Tell: A Neural Image Caption Generator. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298935"},{"key":"ref_43","unstructured":"Kingma, D.P., and Ba, J. (2015). Adam: A Method for Stochastic Optimization. arXiv."},{"key":"ref_44","first-page":"364","article-title":"Convolutional Neural Network With Data Augmentation for SAR Target Recognition","volume":"13","author":"Ding","year":"2016","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"318","DOI":"10.1109\/TPAMI.2018.2858826","article-title":"Focal Loss for Dense Object Detection","volume":"42","author":"Lin","year":"2020","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Duque-Arias, D., Velasco-Forero, S., Deschaud, J.E., Goulette, F., Serna, A., Decenci\u00e8re, E., and Marcotegui, B. (2021, January 8\u201310). On power Jaccard losses for semantic segmentation. Proceedings of the VISAPP 2021: 16th International Conference on Computer Vision Theory and Applications, Virtual.","DOI":"10.5220\/0010304005610568"}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/9\/2085\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:01:37Z","timestamp":1760137297000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/9\/2085"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,4,26]]},"references-count":46,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2022,5]]}},"alternative-id":["rs14092085"],"URL":"https:\/\/doi.org\/10.3390\/rs14092085","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,4,26]]}}}