{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,3]],"date-time":"2026-07-03T23:25:08Z","timestamp":1783121108125,"version":"3.54.6"},"reference-count":54,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2014,1,24]],"date-time":"2014-01-24T00:00:00Z","timestamp":1390521600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/3.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Although a large number of new image classification algorithms have been developed, they are rarely tested with the same classification task. In this research, with  the same Landsat Thematic Mapper (TM) data set and the same classification scheme  over Guangzhou City, China, we tested two unsupervised and 13 supervised classification algorithms, including a number of machine learning algorithms that became popular in remote sensing during the past 20 years. Our analysis focused primarily on the spectral information provided by the TM data. We assessed all algorithms in a per-pixel classification decision experiment and all supervised algorithms in a segment-based experiment. We found that when sufficiently representative training samples were used, most algorithms performed reasonably well. Lack of training samples led to greater classification accuracy discrepancies than classification algorithms themselves. Some algorithms were more tolerable to insufficient (less representative) training samples than others. Many algorithms improved the overall accuracy marginally with per-segment decision making.<\/jats:p>","DOI":"10.3390\/rs6020964","type":"journal-article","created":{"date-parts":[[2014,1,24]],"date-time":"2014-01-24T11:30:50Z","timestamp":1390563050000},"page":"964-983","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":352,"title":["Comparison of Classification Algorithms and Training Sample Sizes in Urban Land Classification with Landsat Thematic Mapper Imagery"],"prefix":"10.3390","volume":"6","author":[{"given":"Congcong","family":"Li","sequence":"first","affiliation":[{"name":"State Key Laboratory of Remote Sensing Science, and College of Global Change and Earth System Science, Beijing Normal University, Beijing 100875, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jie","family":"Wang","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Remote Sensing Science, Institute of Remote Sensing and Digital Earth, Chinese Academy of Sciences, Beijing 100101, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lei","family":"Wang","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Remote Sensing Science, Institute of Remote Sensing and Digital Earth, Chinese Academy of Sciences, Beijing 100101, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Luanyun","family":"Hu","sequence":"additional","affiliation":[{"name":"Ministry of Education Key Laboratory for Earth System Modeling, Center for Earth System Science, Tsinghua University, Beijing 100084, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1513-3765","authenticated-orcid":false,"given":"Peng","family":"Gong","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Remote Sensing Science, Institute of Remote Sensing and Digital Earth, Chinese Academy of Sciences, Beijing 100101, China"},{"name":"Ministry of Education Key Laboratory for Earth System Modeling, Center for Earth System Science, Tsinghua University, Beijing 100084, China"},{"name":"Department of Environmental Science, Policy, and Management, University of California, Berkeley, CA 94720-3114, USA"},{"name":"Joint Center for Global Change Studies, Beijing 100875, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2014,1,24]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"823","DOI":"10.1080\/01431160600746456","article-title":"A survey of image classification methods and techniques for improving classification performance","volume":"28","author":"Lu","year":"2007","journal-title":"Int. J. Remote Sens"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1459","DOI":"10.1080\/01431169208904202","article-title":"Land-use classification of SPOT HRV data using a cover-frequency method","volume":"13","author":"Gong","year":"1992","journal-title":"Int. J. Remote Sens"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"3285","DOI":"10.1080\/01431161003745657","article-title":"Object-based detailed land-cover classification with high spatial resolution IKONOS imagery","volume":"32","author":"Pu","year":"2011","journal-title":"Int. J. Remote Sens"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1920","DOI":"10.1109\/TGRS.2003.814627","article-title":"A hierarchical fuzzy classification approach for high resolution multispectral data over urban areas","volume":"41","author":"Shackelford","year":"2003","journal-title":"IEEE Trans. Geosci. Remote Sens"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"679","DOI":"10.14358\/PERS.75.6.679","article-title":"An adaptive thresholding multiple classifiers system for remote sensing image classification","volume":"75","author":"Tzeng","year":"2009","journal-title":"Photogramm. Eng. Remote Sens"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"433","DOI":"10.1109\/TGRS.2004.837325","article-title":"Results and implications of a study of fifteen years of satellite image classification experiments","volume":"43","author":"Wilkinson","year":"2005","journal-title":"IEEE Trans. Geosci. Remote Sens"},{"key":"ref_7","first-page":"597","article-title":"An assessment of some factors influencing multispectral land-cover classification","volume":"56","author":"Gong","year":"1990","journal-title":"Photogramm. Eng. Remote Sens"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1371","DOI":"10.1080\/13658810802443432","article-title":"Evaluating the temporal and spatial urban expansion patterns of Guangzhou from 1979 to 2003 by remote sensing and GIS methods","volume":"23","author":"Fan","year":"2009","journal-title":"Int. J. Geogr. Inf. Sci"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1223","DOI":"10.3390\/s7071323","article-title":"Land use and land cover change in Guangzhou, China, from 1998 to 2003, based on Landsat TM\/ETM+ imagery","volume":"7","author":"Fan","year":"2007","journal-title":"Sensors"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1985","DOI":"10.1080\/01431160110075532","article-title":"Monitoring land-use change in the Pearl River Delta using Landsat TM","volume":"23","author":"Seto","year":"2002","journal-title":"Int. J. Remote Sens"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"230","DOI":"10.1016\/S0034-4257(00)00169-3","article-title":"Classification and change detection using Landsat TM data: When and how to correct atmospheric effects?","volume":"75","author":"Song","year":"2001","journal-title":"Remote Sens. Environ"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1016\/0034-4257(92)90011-8","article-title":"A comparison of spatial feature extraction algorithms for land-use mapping with SPOT HRV data","volume":"40","author":"Gong","year":"1992","journal-title":"Remote Sens. Environ"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"2607","DOI":"10.1080\/01431161.2012.748992","article-title":"Finer resolution observation and monitoring of global land cover: First mapping results with landsat TM and ETM+ data","volume":"34","author":"Gong","year":"2013","journal-title":"Int. J. Remote Sens"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.rse.2006.03.004","article-title":"Training set size requirements for the classification of a specific class","volume":"104","author":"Foody","year":"2006","journal-title":"Remote Sens. Environ"},{"key":"ref_15","unstructured":"Mather, P.M. (2004). Computer Processing of Remotely-Sensed Images, John Wiley & Sons, Ltd. [3rd ed]."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"685","DOI":"10.1016\/0167-8655(92)90097-J","article-title":"Variability and bias in experimentally measured classifier error rates","volume":"13","author":"Piper","year":"1992","journal-title":"Pattern Recognit. Lett"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"468","DOI":"10.1016\/j.rse.2005.08.011","article-title":"On the relationship between training sample size and data dimensionality: Monte Carlo analysis of broadband multi-temporal classification","volume":"98","author":"McVicar","year":"2005","journal-title":"Remote Sens. Environ"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Congalton, R.G., and Green, K (1999). Assessing the Accuracy of Remotely Sensed Data: Principles and Practices, CRC Press.","DOI":"10.1201\/9781420048568"},{"key":"ref_19","unstructured":"Ball, G.H., and Hall, D.J. (1965). Isodata, a Novel Method of Data Analysis and Pattern Classification, Stanford Research Institute."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"64","DOI":"10.1016\/j.isprsjprs.2013.06.003","article-title":"Clustering based on eigenspace transformation\u2014CBEST for efficient classification","volume":"83","author":"Chen","year":"2013","journal-title":"ISPRS J. Photogramm. Remote Sens"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1007\/BF00153759","article-title":"Instance-based learning algorithms","volume":"6","author":"Aha","year":"1991","journal-title":"Mach. Learn"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"191","DOI":"10.2307\/2347628","article-title":"Ridge estimators in logistic regression","volume":"41","year":"1992","journal-title":"Appl. Stat"},{"key":"ref_23","unstructured":"Quinlan, R (1993). C4.5: Programs for Machine Learning, Morgan Kaufmann Publishers."},{"key":"ref_24","unstructured":"Breiman, L., Friedman, J.H., Olshen, R.A., and Stone, C.J. (1984). Classification and Regression Trees, Wadsworth Press."},{"key":"ref_25","first-page":"815","article-title":"Split selection methods for classification trees","volume":"7","author":"Loh","year":"1997","journal-title":"Stat. Sin"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random forests","volume":"45","author":"Breiman","year":"2001","journal-title":"Mach. Learn"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/1961189.1961199","article-title":"LIBSVM: A library for support vector machines","volume":"2","author":"Chang","year":"2011","journal-title":"ACM Trans. Intell. Syst. Technol"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Bishop, C.M. (1995). Neural Networks for Pattern Recognition, Oxford University Press.","DOI":"10.1093\/oso\/9780198538493.001.0001"},{"key":"ref_29","first-page":"161","article-title":"Logistic model trees","volume":"95","author":"Niels","year":"2005","journal-title":"Mach. Learn"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Sumner, M., Frank, E., and Hall, M.A. (2005, January 3\u20137). Speeding up Logistic Model Tree Induction. Porto, Portugal.","DOI":"10.1007\/11564126_72"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1007\/BF00058655","article-title":"Bagging predictors","volume":"24","author":"Breiman","year":"1996","journal-title":"Mach. Learn"},{"key":"ref_32","unstructured":"Freund, Y., and Robert, E. (1996, January 3\u20136). Schapire: Experiments with a New Boosting Algorithm. Bari, Italy."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"367","DOI":"10.1016\/S0167-9473(01)00065-2","article-title":"Stochastic gradient boosting","volume":"38","author":"Friedman","year":"2002","journal-title":"Comput. Stat. Data Anal"},{"key":"ref_34","first-page":"120","article-title":"The OpenCV library","volume":"25","author":"Bradski","year":"2000","journal-title":"Dr. Dobb\u2019s J. Softw. Tools"},{"key":"ref_35","unstructured":"Witten, I.H., and Frank, E (2005). Data Mining, Practical Machine Learning Tools and Techniques, Morgan Kaufmann Publishers. [2nd ed]."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1145\/1656274.1656278","article-title":"The WEKA data mining software: An update","volume":"11","author":"Hall","year":"2009","journal-title":"SIGKDD Explor"},{"key":"ref_37","unstructured":"Ridgeway, G. Generalized Boosted Models: A Guide to the GBM Package. Available online: https:\/\/r-forge.r-roject.org\/scm\/viewvc.php\/*checkout*\/pkg\/inst\/doc\/gbm.pdf?revision=17&root=gbm&pathrev=18."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"289","DOI":"10.14358\/PERS.76.3.289","article-title":"Accuracy assessment measures for object based image segmentation goodness","volume":"76","author":"Clinton","year":"2010","journal-title":"Photogramm. Eng. Remote Sens"},{"key":"ref_39","unstructured":"Clinton, N. (2010). BerkeleyImageSeg User\u2019s Guide. CiteSeerX, Available online: http:\/\/citeseerx.ist.psu.edu\/viewdoc\/download?doi=10.1.1.173.1324&rep=rep1&type=pdf."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"606","DOI":"10.1109\/JSTSP.2011.2139193","article-title":"A survey of active learning algorithms for supervised remote sensing image classification","volume":"5","author":"Tuia","year":"2011","journal-title":"IEEE J. Sel. Top. Signal Process"},{"key":"ref_41","unstructured":"Campbell, C., Cristianini, N., and Smola, A.J. (July, January 29). Query Learning with Large Margin Classifiers. Stanford, CA, USA."},{"key":"ref_42","unstructured":"Schohn, G., and Cohn, D. (July, January 29). Less is More: Active Learning with Support Vector Machines."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Strobl, C., Malley, J., and Tutz, G. Supplement to \u201cAn Introduction to Recursive Partitioning: Rational, Application, and Characteristics of Classification and Regression Trees, Bagging, and Random Forests\u201d. Available online: http:\/\/supp.apa.org\/psycarticles\/supplemental\/met_14_4_323\/met_14_4_323_supp.html.","DOI":"10.1037\/a0016973"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"5655","DOI":"10.1080\/014311602331291215","article-title":"Integration of object-based and pixel-based classification for mangrove mapping with IKONOS imagery","volume":"25","author":"Wang","year":"2004","journal-title":"Int. J. Remote Sens"},{"key":"ref_45","first-page":"67","article-title":"The use of structural information for improving land-cover classification accuracies at the rural-urban fringe","volume":"56","author":"Gong","year":"1990","journal-title":"Photogramm. Eng. Remote Sens"},{"key":"ref_46","first-page":"423","article-title":"Frequency-based contextual classification and grey-level vector reduction for land-use identification","volume":"58","author":"Gong","year":"1992","journal-title":"Photogramm. Eng. Remote Sens"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"529","DOI":"10.14358\/PERS.69.5.529","article-title":"Comparison of different gray-level reduction schemes for a revised texture spectrum method for land-use classification using IKONOS imagery","volume":"6","author":"Xu","year":"2003","journal-title":"Photogramm. Eng. Remote Sens"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"167","DOI":"10.1016\/j.rse.2005.12.012","article-title":"A spatial-temporal approach to monitoring forest disease spread using multi-temporal high spatial resolution imagery","volume":"101","author":"Liu","year":"2006","journal-title":"Remote Sens. Environ"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"998","DOI":"10.1080\/2150704X.2013.828180","article-title":"A comparison of object-based and contextual pixel-based classifications using high and medium spatial resolution images","volume":"4","author":"Cai","year":"2013","journal-title":"Remote Sens. Lett"},{"key":"ref_50","first-page":"149","article-title":"Object features for pixel-based classification of urban areas comparing different machine learning algorithms","volume":"3","author":"Wolf","year":"2013","journal-title":"Photogramm. Fernerkund"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"579","DOI":"10.1016\/j.isprsjprs.2011.03.002","article-title":"Converting local spectral and spatial information from a priori classifiers into contextual knowledge for impervious surface classification","volume":"66","author":"Luo","year":"2011","journal-title":"ISPRS J. Photogramm. Remote Sens"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1016\/j.rse.2011.02.030","article-title":"Remote sensing of impervious surfaces in the urban areas: Requirements, methods, and trends","volume":"117","author":"Weng","year":"2012","journal-title":"Remote Sens. Environ"},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"570","DOI":"10.1109\/JPROC.2012.2189089","article-title":"Human settlements: A global challenge for EO data processing and interpretation","volume":"101","author":"Gamba","year":"2013","journal-title":"IEEE Proc"},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"129","DOI":"10.1016\/j.isprsjprs.2013.01.011","article-title":"Assessing reference dataset representativeness through confidence metrics based on information density","volume":"78","author":"Mountrakis","year":"2011","journal-title":"ISPRS J. Photogramm. Remote Sens"}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/6\/2\/964\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T21:07:39Z","timestamp":1760216859000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/6\/2\/964"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2014,1,24]]},"references-count":54,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2014,2]]}},"alternative-id":["rs6020964"],"URL":"https:\/\/doi.org\/10.3390\/rs6020964","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2014,1,24]]}}}