{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,5]],"date-time":"2026-07-05T04:42:54Z","timestamp":1783226574145,"version":"3.54.6"},"reference-count":51,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2019,1,30]],"date-time":"2019-01-30T00:00:00Z","timestamp":1548806400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100010198","name":"Ministerio de Econom\u00eda, Industria y Competitividad, Gobierno de Espa\u00f1a","doi-asserted-by":"publisher","award":["TIN2017-88209-C2-2-R"],"award-info":[{"award-number":["TIN2017-88209-C2-2-R"]}],"id":[{"id":"10.13039\/501100010198","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Analyzing land use and land cover (LULC) using remote sensing (RS) imagery is essential for many environmental and social applications. The increase in availability of RS data has led to the development of new techniques for digital pattern classification. Very recently, deep learning (DL) models have emerged as a powerful solution to approach many machine learning (ML) problems. In particular, convolutional neural networks (CNNs) are currently the state of the art for many image classification tasks. While there exist several promising proposals on the application of CNNs to LULC classification, the validation framework proposed for the comparison of different methods could be improved with the use of a standard validation procedure for ML based on cross-validation and its subsequent statistical analysis. In this paper, we propose a general CNN, with a fixed architecture and parametrization, to achieve high accuracy on LULC classification over RS data from different sources such as radar and hyperspectral. We also present a methodology to perform a rigorous experimental comparison between our proposed DL method and other ML algorithms such as support vector machines, random forests, and k-nearest-neighbors. The analysis carried out demonstrates that the CNN outperforms the rest of techniques, achieving a high level of performance for all the datasets studied, regardless of their different characteristics.<\/jats:p>","DOI":"10.3390\/rs11030274","type":"journal-article","created":{"date-parts":[[2019,1,30]],"date-time":"2019-01-30T10:58:27Z","timestamp":1548845907000},"page":"274","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":218,"title":["A Framework for Evaluating Land Use and Land Cover Classification Using Convolutional Neural Networks"],"prefix":"10.3390","volume":"11","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4729-8604","authenticated-orcid":false,"given":"Manuel","family":"Carranza-Garc\u00eda","sequence":"first","affiliation":[{"name":"Division of Computer Science, University of Sevilla, ES-41012 Seville, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1300-4647","authenticated-orcid":false,"given":"Jorge","family":"Garc\u00eda-Guti\u00e9rrez","sequence":"additional","affiliation":[{"name":"Division of Computer Science, University of Sevilla, ES-41012 Seville, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8243-2186","authenticated-orcid":false,"given":"Jos\u00e9 C.","family":"Riquelme","sequence":"additional","affiliation":[{"name":"Division of Computer Science, University of Sevilla, ES-41012 Seville, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2019,1,30]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1016\/j.isprsjprs.2013.03.006","article-title":"Change detection from remotely sensed images: From pixel-based to object-based approaches","volume":"80","author":"Hussain","year":"2013","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"259","DOI":"10.1016\/j.rse.2013.04.005","article-title":"High spatial resolution three-dimensional mapping of vegetation spectral dynamics using computer vision","volume":"136","author":"Dandois","year":"2013","journal-title":"Remote Sens. Environ."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"2886","DOI":"10.1111\/gcb.12553","article-title":"Urban expansion dynamics and natural habitat loss in China: A multiscale landscape perspective","volume":"20","author":"He","year":"2014","journal-title":"Glob. Chang. Biol."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1016\/j.envsoft.2017.06.027","article-title":"An integrated assessment of urban flooding mitigation strategies for robust decision making","volume":"95","author":"Xie","year":"2017","journal-title":"Environ. Model. Softw."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1016\/j.rse.2012.10.017","article-title":"A meta-analysis of terrestrial aboveground biomass estimation using lidar remote sensing","volume":"128","author":"Zolkos","year":"2013","journal-title":"Remote Sens. Environ."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"2868","DOI":"10.1109\/JSTARS.2016.2582921","article-title":"Semantic Labeling of Aerial and Satellite Imagery","volume":"9","author":"Paisitkriangkrai","year":"2016","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"2094","DOI":"10.1109\/JSTARS.2014.2329330","article-title":"Deep learning-based classification of hyperspectral data","volume":"7","author":"Chen","year":"2014","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_8","unstructured":"Jensen, J.R. (2015). Introductory Digital Image Processing: A Remote Sensing Perspective, Prentice Hall Press. [4th ed.]."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1016\/S0034-4257(01)00296-6","article-title":"A comparison of methods for monitoring multitemporal vegetation change using Thematic Mapper imagery","volume":"80","author":"Rogan","year":"2002","journal-title":"Remote Sens. Environ."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"361","DOI":"10.1016\/S0031-3203(99)00215-0","article-title":"A linear constrained distance-based discriminant analysis for hyperspectral image classification","volume":"34","author":"Du","year":"2001","journal-title":"Pattern Recognit."},{"key":"ref_11","first-page":"33","article-title":"A synergistic automatic clustering technique (SYNERACT) for multispectral image Analysis","volume":"68","year":"2002","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"369","DOI":"10.1016\/j.agee.2005.11.013","article-title":"Regional patterns of agricultural land use and deforestation in Colombia","volume":"114","author":"Etter","year":"2006","journal-title":"Agric. Ecosyst. Environ."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"322","DOI":"10.1016\/j.rse.2005.05.008","article-title":"Decision tree regression for soft classification of remote sensing data","volume":"97","author":"Xu","year":"2005","journal-title":"Remote Sens. Environ."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"2112","DOI":"10.1109\/TGRS.2008.916629","article-title":"Supervised Classification of Remotely Sensed Imagery Using a Modified k-NN Technique","volume":"46","author":"Samaniego","year":"2008","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"294","DOI":"10.1016\/j.patrec.2005.08.011","article-title":"Random forests for land cover classification","volume":"27","author":"Gislason","year":"2006","journal-title":"Pattern Recognit. Lett."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"617","DOI":"10.1080\/01431160701352154","article-title":"The application of artificial neural networks to the analysis of remotely sensed data","volume":"29","author":"Mas","year":"2008","journal-title":"Int. J. Remote Sens."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1778","DOI":"10.1109\/TGRS.2004.831865","article-title":"Classification of hyperspectral remote sensing images with support vector machines","volume":"42","author":"Melgani","year":"2004","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1016\/j.rse.2016.02.028","article-title":"A meta-analysis of remote sensing research on supervised pixel-based land-cover image classification processes: General guidelines for practitioners and future research","volume":"177","author":"Khatami","year":"2016","journal-title":"Remote Sens. Environ."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (arXiv, 2015). Deep Residual Learning for Image Recognition, arXiv.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"e1264","DOI":"10.1002\/widm.1264","article-title":"Deep learning for remote sensing image classification: A survey","volume":"8","author":"Li","year":"2018","journal-title":"Wiley Interdiscip. Rev. Data Min. Knowl. Discov."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"778","DOI":"10.1109\/LGRS.2017.2681128","article-title":"Deep Learning Classification of Land Cover and Crop Types Using Remote Sensing Data","volume":"14","author":"Kussul","year":"2017","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_22","first-page":"364","article-title":"Convolutional Neural Network With Data Augmentation for SAR Target Recognition","volume":"13","author":"Ding","year":"2016","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1349","DOI":"10.1109\/TGRS.2015.2478379","article-title":"Unsupervised deep feature extraction for remote sensing image classification","volume":"54","author":"Romero","year":"2016","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_24","unstructured":"Castelluccio, M., Poggi, G., Sansone, C., and Verdoliva, L. (arXiv, 2015). Land Use Classification in Remote Sensing Images by Convolutional Neural Networks, arXiv."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"5632","DOI":"10.1080\/01431161.2016.1246775","article-title":"Stacked Autoencoder-based deep learning for remote-sensing image classification: A case study of African land-cover mapping","volume":"37","author":"Li","year":"2016","journal-title":"Int. J. Remote Sens."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"2381","DOI":"10.1109\/JSTARS.2015.2388577","article-title":"Spectral\u2013Spatial Classification of Hyperspectral Data Based on Deep Belief Network","volume":"8","author":"Chen","year":"2015","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Li, Y., Zhang, H., and Shen, Q. (2017). Spectral\u2013Spatial Classification of Hyperspectral Imagery with 3D Convolutional Neural Network. Remote Sens., 9.","DOI":"10.3390\/rs9010067"},{"key":"ref_28","first-page":"1","article-title":"FMRSS Net: Fast Matrix Representation-Based Spectral-Spatial Feature Learning Convolutional Neural Network for Hyperspectral Image Classification","volume":"2018","author":"Hou","year":"2018","journal-title":"Math. Probl. Eng."},{"key":"ref_29","first-page":"1137","article-title":"A Study of Cross-validation and Bootstrap for Accuracy Estimation and Model Selection","volume":"Volume 2","author":"Kohavi","year":"1995","journal-title":"Proceedings of the 14th International Joint Conference on Artificial Intelligence (IJCAI\u201995)"},{"key":"ref_30","first-page":"2677","article-title":"An extension on \u201cstatistical comparisons of classifiers over multiple data sets\u201d for all pairwise comparisons","volume":"9","author":"Herrera","year":"2008","journal-title":"J. Mach. Learn. Res."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Mahdianpari, M., Salehi, B., Rezaee, M., Mohammadimanesh, F., and Zhang, Y. (2018). Very Deep Convolutional Neural Networks for Complex Land Cover Mapping Using Multispectral Remote Sensing Imagery. Remote Sens., 10.","DOI":"10.3390\/rs10071119"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1016\/j.neunet.2017.07.017","article-title":"A patch-based convolutional neural network for remote sensing image classification","volume":"95","author":"Sharma","year":"2017","journal-title":"Neural Netw."},{"key":"ref_33","unstructured":"Pereira, F., Burges, C.J.C., Bottou, L., and Weinberger, K.Q. (2012). ImageNet Classification with Deep Convolutional Neural Networks. Proceedings of the 25th International Conference on Neural Information Processing Systems, Volume 1 (NIPS\u201912), Curran Associates Inc."},{"key":"ref_34","unstructured":"Simonyan, K., and Zisserman, A. (arXiv, 2014). Very Deep Convolutional Networks for Large-Scale Image Recognition, arXiv."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"6232","DOI":"10.1109\/TGRS.2016.2584107","article-title":"Deep Feature Extraction and Classification of Hyperspectral Images Based on Convolutional Neural Networks","volume":"54","author":"Chen","year":"2016","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_36","unstructured":"F\u00fcrnkranz, J., and Joachims, T. (2010, January 21). Rectified linear units improve Restricted Boltzmann machines. Proceedings of the 27th International Conference on Machine Learning (ICML 2010), Haifa, Israel."},{"key":"ref_37","unstructured":"F\u00fcrnkranz, J., and Joachims, T. (2010, January 21). A Theoretical Analysis of Feature Pooling in Visual Recognition. Proceedings of the 27th International Conference on Machine Learning (ICML 2010), Haifa, Israel."},{"key":"ref_38","first-page":"249","article-title":"Understanding the difficulty of training deep feedforward neural networks","volume":"Volume 9","author":"Teh","year":"2010","journal-title":"Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics"},{"key":"ref_39","unstructured":"Ioffe, S., and Szegedy, C. (arXiv, 2015). Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift, arXiv."},{"key":"ref_40","unstructured":"Hinton, G.E., Srivastava, N., Krizhevsky, A., Sutskever, I., and Salakhutdinov, R. (arXiv, 2012). Improving neural networks by preventing co-adaptation of feature detectors, arXiv."},{"key":"ref_41","unstructured":"Luo, P., Wang, X., Shao, W., and Peng, Z. (arXiv, 2018). Towards Understanding Regularization in Batch Normalization, arXiv."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Li, X., Chen, S., Hu, X., and Yang, J. (arXiv, 2018). Understanding the Disharmony between Dropout and Batch Normalization by Variance Shift, arXiv.","DOI":"10.1109\/CVPR.2019.00279"},{"key":"ref_43","unstructured":"(2018, December 10). GRSS DASE Website Competition. Available online: http:\/\/dase.grss-ieee.org."},{"key":"ref_44","unstructured":"(2019, January 28). 2D Convolutional Neural Networks for Land Use and Land Cover Classification of Radar and Hyperspectral Images. Available online: https:\/\/github.com\/carranza96\/cnn-landcover."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"3735","DOI":"10.1016\/j.csda.2009.04.009","article-title":"Estimating classification error rate: Repeated cross-validation, repeated hold-out and bootstrap","volume":"53","author":"Kim","year":"2009","journal-title":"Comput. Stat. Data Anal."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Hurni, K., Schneider, A., Heinimann, A., Nong, D.H., and Fox, J. (2017). Mapping the Expansion of Boom Crops in Mainland Southeast Asia Using Dense Time Stacks of Landsat Data. Remote Sens., 9.","DOI":"10.3390\/rs9040320"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Graves, S.J., Asner, G.P., Martin, R.E., Anderson, C.B., Colgan, M.S., Kalantari, L., and Bohlman, S.A. (2016). Tree Species Abundance Predictions in a Tropical Agricultural Landscape with a Supervised Classification Model and Imbalanced Data. Remote Sens., 8.","DOI":"10.3390\/rs8020161"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1109\/TIT.1968.1054102","article-title":"On the mean accuracy of statistical pattern recognizers","volume":"14","author":"Hughes","year":"1968","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1007\/s10994-006-6226-1","article-title":"Extremely randomized trees","volume":"63","author":"Geurts","year":"2006","journal-title":"Mach. Learn."},{"key":"ref_50","first-page":"1","article-title":"Statistical Comparisons of Classifiers over Multiple Data Sets","volume":"7","year":"2006","journal-title":"J. Mach. Learn. Res."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Gao, Q., Lim, S., and Jia, X. (2018). Hyperspectral Image Classification Using Convolutional Neural Networks and Multiple Feature Learning. Remote Sens., 10.","DOI":"10.3390\/rs10020299"}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/11\/3\/274\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T12:29:49Z","timestamp":1760185789000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/11\/3\/274"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,1,30]]},"references-count":51,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2019,2]]}},"alternative-id":["rs11030274"],"URL":"https:\/\/doi.org\/10.3390\/rs11030274","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,1,30]]}}}