{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T02:50:02Z","timestamp":1760237402755,"version":"build-2065373602"},"reference-count":32,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2020,4,8]],"date-time":"2020-04-08T00:00:00Z","timestamp":1586304000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100000844","name":"European Space Agency","doi-asserted-by":"publisher","award":["WP-3310-5-1"],"award-info":[{"award-number":["WP-3310-5-1"]}],"id":[{"id":"10.13039\/501100000844","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>A Round Robin exercise was implemented by ESA to compare different classification methods in detecting clouds from images taken by the PROBA-V sensor. A high-quality dataset of 1350 reflectances and Clear\/Cloudy corresponding labels had been prepared by ESA in the framework of the exercise. Motivated by both the experience acquired by one of the authors in this exercise and the availability of such a reliable annotated dataset, we present a full assessment of the methodology proposed therein. Our objective is also to investigate specific issues related to cloud detection when remotely sensed images comprise only a few spectral bands in the visible and near-infrared. For this purpose, we consider a bunch of well-known classification methods. First, we demonstrate the feasibility of using a training dataset semi-automatically obtained from other accurate algorithms. In addition, we investigate the effect of ancillary information, e.g., surface type or climate, on accuracy. Then we compare the different classification methods using the same training dataset under different configurations. We also perform a consensus analysis aimed at estimating the degree of mutual agreement among classification methods in detecting Clear or Cloudy sky conditions.<\/jats:p>","DOI":"10.3390\/s20072090","type":"journal-article","created":{"date-parts":[[2020,4,9]],"date-time":"2020-04-09T03:40:19Z","timestamp":1586403619000},"page":"2090","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Cloud Detection: An Assessment Study from the ESA Round Robin Exercise for PROBA-V"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1482-4898","authenticated-orcid":false,"given":"Umberto","family":"Amato","sequence":"first","affiliation":[{"name":"Istituto di Scienze Applicate e Sistemi Intelligenti \u2018E. Caianiello\u2019 CNR, 80131 Napoli, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Anestis","family":"Antoniadis","sequence":"additional","affiliation":[{"name":"Laboratoire Jean Kuntzmann, Department of Statistics, Universit\u00e9 Joseph Fourier, 38000 Grenoble, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4570-1690","authenticated-orcid":false,"given":"Maria Francesca","family":"Carfora","sequence":"additional","affiliation":[{"name":"Istituto per le Applicazioni del Calcolo \u2018Mauro Picone\u2019 CNR, 80100 Napoli, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,4,8]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Masiello, G., Serio, C., Venafra, S., Liuzzi, G., Poutier, L., and Goettsche, F. (2018). Physical retrieval of land surface emissivity spectra from hyper-spectra infrared observations and validation with in situ measurements. Remote Sens., 10.","DOI":"10.3390\/rs10060976"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"6576","DOI":"10.1364\/AO.55.006576","article-title":"Demonstration of random projections applied to the retrieval problem of geophysical parameters from hyper-spectral infrared observations","volume":"55","author":"Serio","year":"2016","journal-title":"Appl. Opt."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1016\/j.rse.2017.03.026","article-title":"Cloud detection algorithm comparison and validation for operational Landsat data products","volume":"194","author":"Foga","year":"2017","journal-title":"Remote Sens. Environ."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Iannone, R.Q., Niro, F., Goryl, P., Dransfeld, S., Hoersch, B., Stelzer, K., Kirches, G., Paperin, M., Brockmann, C., and G\u00f3mez-Chova, L. (2017, January 27\u201329). Proba-V cloud detection Round Robin: Validation results and recommendations. Proceedings of the 2017 9th International Workshop on the Analysis of Multitemporal Remote Sensing Images (MultiTemp), Brugge, Belgium.","DOI":"10.1109\/Multi-Temp.2017.8035219"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"3355","DOI":"10.5194\/amt-7-3355-2014","article-title":"Cloud mask via cumulative discriminant analysis applied to satellite infrared observations: Scientific basis and initial evaluation","volume":"7","author":"Amato","year":"2014","journal-title":"Atmos. Meas. Tech."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"247","DOI":"10.1016\/j.rse.2019.03.039","article-title":"A cloud detection algorithm for satellite imagery based on deep learning","volume":"229","author":"Jeppesen","year":"2019","journal-title":"Remote Sens. Environ."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1349","DOI":"10.1080\/01431161.2019.1667548","article-title":"Satellite data cloud detection using deep learning supported by hyperspectral data","volume":"41","author":"Sun","year":"2019","journal-title":"Int. J. Remote Sens."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"750","DOI":"10.1016\/j.rse.2007.06.004","article-title":"Statistical cloud detection from SEVIRI multispectral images","volume":"112","author":"Amato","year":"2008","journal-title":"Remote Sens. Environ."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"2589","DOI":"10.1080\/01431161.2014.883097","article-title":"PROBA-V mission for global vegetation monitoring: Standard products and image quality","volume":"35","author":"Dierckx","year":"2014","journal-title":"Int. J. Remote Sens."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Tot\u00e9, C., Swinnen, E., Sterckx, S., Adriaensen, S., Benhadj, I., Iordache, M.D., Bertels, L., Kirches, G., Stelzer, K., and Dierckx, W. (2018). Evaluation of PROBA-V Collection 1: Refined Radiometry, Geometry, and Cloud Screening. Remote Sens., 10.","DOI":"10.3390\/rs10091375"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"4707","DOI":"10.1080\/01431160500166128","article-title":"MSG\/SEVIRI cloud mask and type from SAFNWC","volume":"26","author":"Derrien","year":"2005","journal-title":"Int. J. Remote Sens."},{"key":"ref_12","unstructured":"Ackerman, S.A., and Frey, R. (2020, March 25). MODIS Atmosphere L2 Cloud Mask Product. Available online: http:\/\/dx.doi.org\/10.5067\/MODIS\/MOD35_L2.006."},{"key":"ref_13","unstructured":"Ackerman, S.A., and Frey, R. (2020, March 25). MODIS Atmosphere L2 Cloud Mask Product. Available online: http:\/\/dx.doi.org\/10.5067\/MODIS\/MYD35_L2.006."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1303","DOI":"10.1080\/014311600210191","article-title":"Development of a global land cover characteristics database and IGBP DISCover from 1 km AVHRR data","volume":"21","author":"Loveland","year":"2000","journal-title":"Int. J. Remote Sens."},{"key":"ref_15","unstructured":"(2020, March 25). METI (Ministry of Economy Trade and Industry of Japan) and NASA (US National Aeronautics and Space Administration), Available online: https:\/\/asterweb.jpl.nasa.gov\/gdem.asp."},{"key":"ref_16","unstructured":"Copernicus Climate Change Service (2019, October 04). C3S ERA5-Land reanalysis. Available online: https:\/\/cds.climate.copernicus.eu\/cdsapp#!\/home."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1529","DOI":"10.3390\/rs70201529","article-title":"Multilayer perceptron neural networks model for meteosat second generation SEVIRI daytime cloud masking","volume":"7","author":"Taravat","year":"2015","journal-title":"Remote Sens."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"62","DOI":"10.1016\/j.rse.2018.09.029","article-title":"New neural network cloud mask algorithm based on radiative transfer simulations","volume":"219","author":"Chen","year":"2018","journal-title":"Remote Sens. Environ."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"5530","DOI":"10.1080\/01431161.2019.1580788","article-title":"Energy-based cloud detection in multispectral images based on the SVM technique","volume":"40","author":"Sui","year":"2019","journal-title":"Int. J. Remote Sens."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Hollstein, A., Segl, K., Guanter, L., Brell, M., and Enesco, M. (2016). Ready-to-use methods for the detection of clouds, cirrus, snow, shadow, water and clear sky pixels in Sentinel-2 MSI images. Remote Sens., 8.","DOI":"10.3390\/rs8080666"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"588","DOI":"10.1016\/j.isprsjprs.2011.03.005","article-title":"A\u2018cloud mask methodology for high resolution remote sensing data combining information from high and medium resolution optical sensors","volume":"66","author":"Sedano","year":"2011","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"197","DOI":"10.1016\/j.isprsjprs.2019.02.017","article-title":"Deep learning based cloud detection for medium and high resolution remote sensing images of different sensors","volume":"150","author":"Li","year":"2019","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Francis, A., Sidiropoulos, P., and Muller, J.P. (2019). CloudFCN: Accurate and Robust Cloud Detection for Satellite Imagery with Deep Learning. Remote Sens., 11.","DOI":"10.3390\/rs11192312"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"124","DOI":"10.1016\/j.isprsjprs.2019.08.018","article-title":"Deep learning for multi-modal classification of cloud, shadow and land cover scenes in PlanetScope and Sentinel-2 imagery","volume":"157","author":"Shendryk","year":"2019","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_25","first-page":"735","article-title":"Independent component discriminant analysis","volume":"3","author":"Amato","year":"2003","journal-title":"Int. J. Math."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1007\/BF02478259","article-title":"A logical calculus of the ideas immanent in nervous activity","volume":"5","author":"McCulloch","year":"1943","journal-title":"Bull. Math. Biophys."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"386","DOI":"10.1037\/h0042519","article-title":"The perceptron: A probabilistic model for information storage and organization in the brain","volume":"65","author":"Rosenblatt","year":"1958","journal-title":"Psychol. Rev."},{"key":"ref_28","unstructured":"Duda, R., Hart, P., and Stork, D. (2012). Pattern Classification, John Wiley & Sons."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"595","DOI":"10.1214\/aos\/1176343886","article-title":"Consistent Nonparametric Regression","volume":"5","author":"Stone","year":"1977","journal-title":"Ann. Stat."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"259","DOI":"10.1111\/j.2517-6161.1986.tb01412.x","article-title":"On the Statistical Analysis of Dirty Pictures","volume":"48","author":"Besag","year":"1986","journal-title":"J. R. Stat. Soc. Ser. B (Methodol.)"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"4966","DOI":"10.1016\/j.csda.2008.04.015","article-title":"Localized empirical discriminant analysis","volume":"52","author":"Cutillo","year":"2008","journal-title":"Comput. Stat. Data Anal."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Zi, Y., Xie, F., and Jiang, Z. (2018). A Cloud Detection Method for Landsat 8 Images Based on PCANet. Remote Sens., 10.","DOI":"10.3390\/rs10060877"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/7\/2090\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:16:28Z","timestamp":1760174188000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/7\/2090"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,4,8]]},"references-count":32,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2020,4]]}},"alternative-id":["s20072090"],"URL":"https:\/\/doi.org\/10.3390\/s20072090","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2020,4,8]]}}}