{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T20:10:26Z","timestamp":1760213426475,"version":"build-2065373602"},"reference-count":66,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2016,9,13]],"date-time":"2016-09-13T00:00:00Z","timestamp":1473724800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001807","name":"Funda\u00e7\u00e3o de Amparo \u00e0 Pesquisa do Estado de S\u00e3o Paulo","doi-asserted-by":"publisher","award":["2011\/05975-4"],"award-info":[{"award-number":["2011\/05975-4"]}],"id":[{"id":"10.13039\/501100001807","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003593","name":"Conselho Nacional de Desenvolvimento Cient\u00edfico e Tecnol\u00f3gico","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100003593","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["IJGI"],"abstract":"<jats:p>Landslide scar location is fundamental for the risk management process, e.g., it allows mitigation of these areas, decreasing the associated hazards for the population. Remote sensing data usage is an essential tool for landslide identification, mapping, and monitoring. Despite its potential use for landslide risk management, remote sensing usage does have a few drawbacks. The aforementioned events commonly occur at high steep slope regions, frequently associated with shadow occurrence in satellite images, which impairs the identification process and results in low accuracy classifications. In this sense, this paper aims to evaluate the accuracy of different ensembles of multiple classifier systems (MCSs) for landslide scar identification. A severe landslide event on a steep slope with a high rainfall rate area in the southeast region of Brazil was chosen. Ten supervised classifiers were used to identify this severe event and other possible features for the LANDSAT thematic mapper (TM) from June of 2000. The results were evaluated, and nine MCSs were constructed based on the accuracy of the classifiers. Voting was applied through the ensemble method, coupled with contextual analysis and random selection tie-breaker methods. Accuracy was evaluated for each classification ensemble, and a progressive enhancement in the ensemble accuracy was noted as the least accurate classifiers were removed. The best accuracy for landslide identification emerged from the ensemble of the three most accurate classification results. In summary, MCS application generally improved the classification quality and led to fewer omission errors, coupled with a better classification percentage for the \u2018landslide\u2019 class. However, the MCS ensemble algorithm selection must be customized to the purpose of the classification. It is crucial to assess single accuracy indicators of each algorithm to ascertain those with the most consistent performance regarding the final results.<\/jats:p>","DOI":"10.3390\/ijgi5090164","type":"journal-article","created":{"date-parts":[[2016,9,13]],"date-time":"2016-09-13T10:10:00Z","timestamp":1473761400000},"page":"164","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Evaluation of Multiple Classifier Systems for Landslide Identification in LANDSAT Thematic Mapper (TM) Images"],"prefix":"10.3390","volume":"5","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1094-1424","authenticated-orcid":false,"given":"Luiz","family":"Manfr\u00e9","sequence":"first","affiliation":[{"name":"Transportation Engineering Department of Polytechnic College, University of Sao Paulo, S\u00e3o Paulo 05507-070, Brazil"}]},{"given":"Rodrigo","family":"De Albuquerque N\u00f3brega","sequence":"additional","affiliation":[{"name":"Cartography Department of Geosciences, Institute of Federal, University of Minas Gerais, Belo Horizonte 31270-901, Brazil"}]},{"given":"Jos\u00e9","family":"Quintanilha","sequence":"additional","affiliation":[{"name":"Department of Transportation Engineering, Polytechnic School of the University of Sao Paulo, S\u00e3o Paulo 05507-070, Brazil"}]}],"member":"1968","published-online":{"date-parts":[[2016,9,13]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"194","DOI":"10.1016\/j.jhydrol.2014.05.022","article-title":"Hydrogeomorphic response to extreme rainfall in headwater systems: Flash floods and debris flow","volume":"518","author":"Borga","year":"2014","journal-title":"J. Hydrol."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"927","DOI":"10.1130\/G33217.1","article-title":"Global pattern of loss of life from landslides","volume":"40","author":"Petley","year":"2012","journal-title":"Geology"},{"key":"ref_3","unstructured":"Wang, H., Zhang, Y., and Hu, H. (2012, January 8\u201320). A Study on the relationship between the occurrence of Landslides and Rainfall. Proceedings of the 2nd International Conference on Electric Technology and Civil Engineering, Yichang, China."},{"key":"ref_4","first-page":"1049","article-title":"Relationships between landslides phenomena and road network: An example from hilly region of Asti Province (North-Western Italy)","volume":"2","author":"Masciocco","year":"2014","journal-title":"Eng. Geol. Soc. Territ."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"389","DOI":"10.1007\/978-3-319-04999-1_55","article-title":"Highway vs. landslides and their consequences in Himalaya","volume":"1","author":"Kumar","year":"2014","journal-title":"Landslide Sci. Safer Geoenvironment"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"273","DOI":"10.1016\/S0169-555X(01)00138-6","article-title":"The effects of logging on frequency and distribution of landslides in three watersheds on Vancouver Island, British Columbia","volume":"43","author":"Guthrie","year":"2002","journal-title":"Geomorphology"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"42","DOI":"10.1016\/j.earscirev.2012.02.001","article-title":"Landslide inventory maps: New tools for an old problem","volume":"112","author":"Guzzetti","year":"2012","journal-title":"Earth Sci. Rev."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1835","DOI":"10.5194\/nhess-14-1835-2014","article-title":"Brief Communication: Rapid mapping of landslide events: The 3 December 2013 Montescaglioso landslide, Italy","volume":"14","author":"Manconi","year":"2014","journal-title":"Nat. Hazards Earth Syst. Sci."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"301","DOI":"10.1007\/978-3-319-05050-8_47","article-title":"Introduction: Remote sensing techniques for landslide mapping and monitoring","volume":"1","author":"Tofani","year":"2014","journal-title":"Landslide Sci. Safer Geoenvironment"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1913","DOI":"10.1080\/01431160512331314047","article-title":"Satellite remote sensing for detailed landslide inventories using change detection and image fusion","volume":"26","author":"Nichol","year":"2005","journal-title":"Int. J. Remote Sens."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Liu, J.G., and Mason, P. (2009). Essential image processing and GIS for Remote Sensing, Wiley-Blackwell.","DOI":"10.1002\/9781118687963"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Lu, D., Weng, Q., Moran, E., Li, G., and Hetrick, S. (2011). Advances in Environmental Remote Sensing: Sensors, Algorithms, and Applications, Part II, CRC Press\/Taylor & Francis Group Publishers.","DOI":"10.1201\/b10599"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Richards, J.A. (1999). Remote Sensing Digital Image Analysis, Springer.","DOI":"10.1007\/978-3-662-03978-6"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Schott, J.R. (2007). Remote Sensing, Oxford University Press.","DOI":"10.1093\/oso\/9780195178173.001.0001"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"545","DOI":"10.1016\/S0034-4257(00)00145-0","article-title":"Combining multiple classifiers: An application using spatial and remotely sensed information for land cover type mapping","volume":"74","author":"Steele","year":"2000","journal-title":"Remote Sens. Environ."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"2291","DOI":"10.1109\/TGRS.2002.802476","article-title":"Multiple classifiers applied to multisource remote sensing data","volume":"40","author":"Briem","year":"2002","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_17","unstructured":"Benediktsson, J.A., Chanussot, J., and Fauvel, M. (2007). Multiple Classifier Systems, Springer."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"4609","DOI":"10.1080\/01431160701244872","article-title":"Increasing soft classification accuracy through the use of an ensemble of classifiers","volume":"28","author":"Doan","year":"2007","journal-title":"Int. J. Remote Sens."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1733","DOI":"10.1080\/01431160600962566","article-title":"Mapping a specific class with an ensemble of classifiers","volume":"28","author":"Foody","year":"2007","journal-title":"Int. J. Remote Sens."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"2880","DOI":"10.1109\/TGRS.2010.2041784","article-title":"Sensitivity of support vector machines to random feature selection in classification of hyperspectral data","volume":"48","author":"Waske","year":"2010","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1461","DOI":"10.1080\/014311699212560","article-title":"Quality assessment of image classification algorithms for land-cover mapping: A review and a proposal for a cost-based approach","volume":"20","author":"Smits","year":"1999","journal-title":"Int. J. Remote Sens."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1016\/S0167-8655(00)00096-9","article-title":"An approach to the automatic design of multiple classifier systems","volume":"22","author":"Giacinto","year":"2001","journal-title":"Pattern Recognit. Lett."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1023\/A:1022859003006","article-title":"Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy","volume":"51","author":"Kuncheva","year":"2003","journal-title":"Mach. Learn."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1007\/BF00058655","article-title":"Bagging predictors","volume":"24","author":"Breiman","year":"1996","journal-title":"Mach. Learn."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"418","DOI":"10.1109\/21.155943","article-title":"Methods of combining multiple classifiers and their applications to handwriting recognition","volume":"22","author":"Xu","year":"1992","journal-title":"IEEE Trans. Syst. Man Cybern."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"292","DOI":"10.1007\/s100440050038","article-title":"Serial combination of multiple experts: A unified evaluation","volume":"2","author":"Rahman","year":"1999","journal-title":"Pattern Anal. Appl."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"4764","DOI":"10.3390\/s120404764","article-title":"Multiple classifier system for remote sensing image classification: A review","volume":"12","author":"Du","year":"2012","journal-title":"Sensors"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Abe, B.T., Olugbara, O.O., and Marwala, T. (2014). Classification of Hyperspectral Images Using Machine Learning Methods, Springer Netherlands.","DOI":"10.1007\/978-94-007-6818-5_39"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"4407","DOI":"10.1080\/01431161.2011.552923","article-title":"Death to Kappa: Birth of quantity disagreement and allocation disagreement for accuracy assessment","volume":"32","author":"Millones","year":"2011","journal-title":"Int. J. Remote Sens."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"604","DOI":"10.1080\/15481603.2013.865399","article-title":"Analyst variation associated with land cover image classification of Landsat ETM+ data for the assessment of coarse spatial resolution regional\/global land cover products","volume":"50","author":"Iiames","year":"2013","journal-title":"GIScience Remote Sens."},{"key":"ref_31","unstructured":"Ogura, A.T. (2006). Rela\u00e7\u00e3o e Previsibilidade de Eventos Meteorol\u00f3gicos Extremos Deflagadores de Acidentes de Movimentos de Massa na Serra do Mar, IPT (Instituto de Pesquisa e Tecnologia)."},{"key":"ref_32","unstructured":"Manfre, L.A. (2015). Identifica\u00e7\u00e3o e Mapeamento de \u00c1reas de Deslizamento Associadas a Rodovias Utilizando Imagens de Sensoriamento Remoto. [Ph.D. Thesis, Escola Polit\u00e9cnica da Universidade de S\u00e3o Paulo]."},{"key":"ref_33","unstructured":"Wolle, C.M. (1988). An\u00e1lise dos Escorregamentos Translacionais Numa Regi\u00e3o da Serra do Mar no Contexto de uma Classifica\u00e7\u00e3o de Mecanismos de Instabiliza\u00e7\u00e3o de Encostas. [Ph.D. Thesis, Escola Polit\u00e9cnica da Universidade de S\u00e3o Paulo]."},{"key":"ref_34","unstructured":"Toledo, J.M.A. (2008). Crop Discrimination Using Harmonic Analysis of EVI Modis Time-Series Data. [Master\u2019s Thesis, Instituto Nacional de Pesquisas Espaciais]."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Ma, H., Qin, Q., and Shen, X. (2008, January 7\u201311). Shadow segmentation and compensation in high resolution satellite images. Proceedings of the International Geoscience and Remote Sensing Symposium, Boston, MA, USA.","DOI":"10.1109\/IGARSS.2008.4779175"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"881","DOI":"10.1109\/TPAMI.2002.1017616","article-title":"An efficient k-means clustering algorithm: Analysis and implementation","volume":"24","author":"Kanungo","year":"2002","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"2687","DOI":"10.1080\/01431160310001618428","article-title":"A hybrid approach to urban land use\/cover mapping using Landsat 7 Enhanced Thematic Mapper Plus (ETM+) images","volume":"25","author":"Lo","year":"2004","journal-title":"Int. J. Remote Sens."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"125","DOI":"10.1016\/j.isprsjprs.2006.09.004","article-title":"Multiple support vector machines for land cover change detection: An application for mapping urban extensions","volume":"61","author":"Nemmour","year":"2006","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_39","first-page":"S27","article-title":"Land cover change assessment using decision trees, support vector machines and maximum likelihood classification algorithms","volume":"12","author":"Otukei","year":"2010","journal-title":"Int. J. Appl. Earth Obs. Geoinf."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"1967","DOI":"10.3390\/s100301967","article-title":"A comparison of spectral angle mapper and artificial neural network classifiers combined with Landsat TM imagery analysis for obtaining burnt area mapping","volume":"10","author":"Petropoulos","year":"2010","journal-title":"Sensors"},{"key":"ref_41","first-page":"1","article-title":"Multi-classifier systems: Review and a roadmap for developers","volume":"3","author":"Ranawana","year":"2006","journal-title":"Int. J. Hybrid Intell. Syst."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"1118","DOI":"10.1175\/1520-0450(2001)040<1118:TCONDT>2.0.CO;2","article-title":"The concept of \u201cnormalized\u201d distribution to describe raindrop spectra: A tool for cloud physics and cloud remote sensing","volume":"40","author":"Testud","year":"2001","journal-title":"J. Appl. Meteorol."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"1761","DOI":"10.1016\/j.energy.2006.11.010","article-title":"Predicting electricity energy consumption: A comparison of regression analysis, decision tree and neural networks","volume":"32","author":"Tso","year":"2007","journal-title":"Energy"},{"key":"ref_44","first-page":"93","article-title":"Decision tree classification of remotely sensed satellite data using spectral separability matrix","volume":"1","author":"Ghose","year":"2010","journal-title":"Int. J. Adv. Comput. Sci. Appl."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Wu, F., Zhan, J., Yan, H., Shi, C., and Huang, J. (2013). Land cover mapping based on multisource spatial data mining approach for climate simulation: A case study in the farming-pastoral ecotone of North China. Adv. Meteorol., 2013.","DOI":"10.1155\/2013\/520803"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"2","DOI":"10.1016\/j.isprsjprs.2009.06.004","article-title":"Object based image analysis for remote sensing","volume":"65","author":"Blaschke","year":"2010","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1016\/0034-4257(91)90048-B","article-title":"A review of assessing the accuracy of classifications of remotely sensed data","volume":"37","author":"Congalton","year":"1999","journal-title":"Remote Sens. Environ."},{"key":"ref_48","unstructured":"Ho, T.K., Hull, J.J., and Srihari, S.N. (September, January 30). On multiple classifier systems for pattern recognition. Proceedings of the 11th IEEE International Conference on Pattern Recognition, Hague, The Netherlands."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"849","DOI":"10.1109\/TFUZZ.2012.2187062","article-title":"Adaptive local fusion with fuzzy integrals","volume":"20","author":"Frigui","year":"2012","journal-title":"IEEE Trans. Fuzzy Syst."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"400","DOI":"10.1016\/j.cities.2005.05.009","article-title":"Analysis of spatio-temporal dynamic pattern and driving forces of urban land in China in 1990s using TM images and GIS","volume":"22","author":"Tian","year":"2005","journal-title":"Cities"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"3440","DOI":"10.1109\/TGRS.2012.2183876","article-title":"A complete processing chain for shadow detection and reconstruction in VHR images","volume":"50","author":"Lorenzi","year":"2012","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_52","first-page":"172","article-title":"Method for landslides identification at the Sao Paulo state coast, Brazil","volume":"33","author":"Shinohara","year":"2014","journal-title":"Geoci\u00eancias"},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1016\/j.inffus.2013.04.006","article-title":"A survey of multiple classifier systems as hybrid systems","volume":"16","author":"Corchado","year":"2014","journal-title":"Inf. Fusion"},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"226","DOI":"10.1109\/34.667881","article-title":"On combining classifiers","volume":"20","author":"Kittler","year":"2006","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1016\/j.neucom.2013.01.052","article-title":"Optimal selection of ensemble classifiers using measures of competence and diversity of base classifiers","volume":"126","author":"Lysiak","year":"2014","journal-title":"Neurocomputing"},{"key":"ref_56","unstructured":"Moreno-Seco, F., Inesta, J., De Leon, P., and Mico, L. (2006). Structural, Syntactic, and Statistical Pattern Recognition, Springer."},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1016\/S0034-4257(01)00295-4","article-title":"Status of land cover classification accuracy assessment","volume":"80","author":"Foody","year":"2002","journal-title":"Remote Sens. Environ."},{"key":"ref_58","first-page":"231","article-title":"Neural network ensembles, cross validation, and active learning","volume":"7","author":"Krogh","year":"1995","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1016\/j.inffus.2004.04.004","article-title":"Diversity creation methods: A survey and categorisation","volume":"6","author":"Brown","year":"2005","journal-title":"Inf. Fusion"},{"key":"ref_60","first-page":"2539","article-title":"The effect of classifier agreement on the accuracy of the combined classifier in decision level fusion","volume":"39","author":"Michail","year":"2005","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"686","DOI":"10.1016\/j.neucom.2005.12.014","article-title":"Evolving hybrid ensembles of learning machines for better generalisation","volume":"69","author":"Chandra","year":"2006","journal-title":"Neurocomputing"},{"key":"ref_62","doi-asserted-by":"crossref","first-page":"1658","DOI":"10.1016\/j.rse.2009.03.014","article-title":"Classification accuracy comparison: Hypothesis tests and the use of confidence intervals in evaluations of difference, equivalence and non-inferiority","volume":"113","author":"Foody","year":"2009","journal-title":"Remote Sens. Environ."},{"key":"ref_63","doi-asserted-by":"crossref","first-page":"420","DOI":"10.1049\/el:20000374","article-title":"Selection of image classifiers","volume":"36","author":"Giacinto","year":"2000","journal-title":"Electron. Lett."},{"key":"ref_64","doi-asserted-by":"crossref","unstructured":"Koc-San, D. (2013). Evaluation of different classification techniques for the detection of glass and plastic greenhouses from WorldView-2 satellite imagery. J. Appl. Remote Sens., 7.","DOI":"10.1117\/1.JRS.7.073553"},{"key":"ref_65","doi-asserted-by":"crossref","first-page":"192","DOI":"10.1016\/j.envsoft.2012.01.014","article-title":"imager\u2014A user-oriented implementation for remote sensing image analysis with Random Forests","volume":"35","author":"Waske","year":"2012","journal-title":"Environ. Model. Softw."},{"key":"ref_66","doi-asserted-by":"crossref","first-page":"083691","DOI":"10.1117\/1.JRS.8.083691","article-title":"Using phenological metrics and the multiple classifier fusion method to map land cover types","volume":"8","author":"Liu","year":"2014","journal-title":"J. Appl. Remote Sens."}],"container-title":["ISPRS International Journal of Geo-Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2220-9964\/5\/9\/164\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T19:30:47Z","timestamp":1760211047000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2220-9964\/5\/9\/164"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,9,13]]},"references-count":66,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2016,9]]}},"alternative-id":["ijgi5090164"],"URL":"https:\/\/doi.org\/10.3390\/ijgi5090164","relation":{},"ISSN":["2220-9964"],"issn-type":[{"type":"electronic","value":"2220-9964"}],"subject":[],"published":{"date-parts":[[2016,9,13]]}}}