{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,14]],"date-time":"2026-05-14T14:24:42Z","timestamp":1778768682324,"version":"3.51.4"},"reference-count":52,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2016,11,1]],"date-time":"2016-11-01T00:00:00Z","timestamp":1477958400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"EU COST","award":["COST Action TD1202"],"award-info":[{"award-number":["COST Action TD1202"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["IJGI"],"abstract":"<jats:p>The accuracy of a map is dependent on the reference dataset used in its construction. Classification analyses used in thematic mapping can, for example, be sensitive to a range of sampling and data quality concerns. With particular focus on the latter, the effects of reference data quality on land cover classifications from airborne thematic mapper data are explored. Variations in sampling intensity and effort are highlighted in a dataset that is widely used in mapping and modelling studies; these may need accounting for in analyses. The quality of the labelling in the reference dataset was also a key variable influencing mapping accuracy. Accuracy varied with the amount and nature of mislabelled training cases with the nature of the effects varying between classifiers. The largest impacts on accuracy occurred when mislabelling involved confusion between similar classes. Accuracy was also typically negatively related to the magnitude of mislabelled cases and the support vector machine (SVM), which has been claimed to be relatively insensitive to training data error, was the most sensitive of the set of classifiers investigated, with overall classification accuracy declining by 8% (significant at 95% level of confidence) with the use of a training set containing 20% mislabelled cases.<\/jats:p>","DOI":"10.3390\/ijgi5110199","type":"journal-article","created":{"date-parts":[[2016,11,1]],"date-time":"2016-11-01T11:01:44Z","timestamp":1477998104000},"page":"199","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":78,"title":["The Sensitivity of Mapping Methods to Reference Data Quality: Training Supervised Image Classifications with Imperfect Reference Data"],"prefix":"10.3390","volume":"5","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6464-3054","authenticated-orcid":false,"given":"Giles","family":"Foody","sequence":"first","affiliation":[{"name":"School of Geography, University of Nottingham, Nottingham, NG7 2RD, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mahesh","family":"Pal","sequence":"additional","affiliation":[{"name":"Department of Civil Engineering, National Institute of Technology, Kurukshetra, Haryana 136119, India"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0087-0594","authenticated-orcid":false,"given":"Duccio","family":"Rocchini","sequence":"additional","affiliation":[{"name":"Department of Biodiversity and Molecular Ecology, Research and Innovation Centre, Fondazione Edmund Mach, Via E. Mach 1, 38010 San Michele all\u2019dige TN, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Carol","family":"Garzon-Lopez","sequence":"additional","affiliation":[{"name":"Ecology and Dynamics of Human-Influenced Systems Research Unit (EDYSAN, FRE 3498 CNRS), University of Picardy Jules Verne, 1 rue des Louvels, FR-80037 Amiens Cedex 1, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1321-0800","authenticated-orcid":false,"given":"Lucy","family":"Bastin","sequence":"additional","affiliation":[{"name":"School of Engineering and Applied Science, Aston University, Birmingham, B4 7ET, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2016,11,1]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"63","DOI":"10.1016\/j.ecoser.2015.07.006","article-title":"Land use mapping error introduces strongly-localised, scale-dependent uncertainty into land use and ecosystem services modelling","volume":"15","author":"Dong","year":"2015","journal-title":"Ecosyst. Serv."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1016\/j.ecolecon.2015.01.003","article-title":"Valuing map validation: The need for rigorous land cover map accuracy assessment in economic valuations of ecosystem services","volume":"111","author":"Foody","year":"2015","journal-title":"Ecol. Econ."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"2496","DOI":"10.3390\/ijgi4042496","article-title":"Impacts of species misidentification on species distribution modeling with presence-only data","volume":"4","author":"Costa","year":"2015","journal-title":"ISPRS Int. J. Geo-Inf."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"783","DOI":"10.1080\/2150704X.2013.798708","article-title":"Ground reference data error and the MIS-estimation of the area of land cover change as a function of its abundance","volume":"4","author":"Foody","year":"2013","journal-title":"Remote Sens. Lett."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"433","DOI":"10.1109\/TGRS.2004.837325","article-title":"Results and implications of a study of fifteen years of satellite image classification experiments","volume":"43","author":"Wilkinson","year":"2005","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1016\/S0034-4257(01)00295-4","article-title":"Status of land cover classification accuracy assessment","volume":"80","author":"Foody","year":"2002","journal-title":"Remote Sens. Environ."},{"key":"ref_7","unstructured":"Global Biodiversity Information Facility (GBIF). Available online: http:\/\/www.gbif.org."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1016\/j.rse.2003.12.007","article-title":"Sources of error in accuracy assessment of thematic land-cover maps in the Brazilian Amazon","volume":"90","author":"Powell","year":"2004","journal-title":"Remote Sens. Environ."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"101","DOI":"10.1023\/A:1021441331839","article-title":"Assessing species misidentification rates through quality assurance of vegetation monitoring","volume":"165","author":"Scott","year":"2003","journal-title":"Plant Ecol."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/s10708-007-9111-y","article-title":"Citizens as sensors: The world of volunteered geography","volume":"69","author":"Goodchild","year":"2007","journal-title":"GeoJournal"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"239","DOI":"10.1007\/s10708-008-9190-4","article-title":"Whither VGI?","volume":"72","author":"Goodchild","year":"2008","journal-title":"GeoJournal"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"847","DOI":"10.1111\/tgis.12033","article-title":"Assessing the accuracy of volunteered geographic information arising from multiple contributors to an internet based collaborative project","volume":"17","author":"Foody","year":"2013","journal-title":"Trans. GIS"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"4831","DOI":"10.1080\/01431160802672864","article-title":"Effect of errors in ground truth on classification accuracy","volume":"30","author":"Carlotto","year":"2009","journal-title":"Int. J. Remote Sens."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"2271","DOI":"10.1016\/j.rse.2010.05.003","article-title":"Assessing the accuracy of land cover change with imperfect ground reference data","volume":"14","author":"Foody","year":"2010","journal-title":"Remote Sens. Environ."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"3965","DOI":"10.3390\/rs6053965","article-title":"Automated training sample extraction for global land cover mapping","volume":"6","author":"Radoux","year":"2014","journal-title":"Remote Sens."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"2142","DOI":"10.1109\/TGRS.2008.2011983","article-title":"A novel context-sensitive semisupervised SVM classifier robust to mislabelled training samples","volume":"47","author":"Bruzzone","year":"2009","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_17","first-page":"249","article-title":"Supervised machine learning: A review of classification techniques","volume":"31","author":"Kotsiantis","year":"2007","journal-title":"Informatica"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"451","DOI":"10.14358\/PERS.81.6.451","article-title":"Integrating user needs on misclassification error sensitivity into image segmentation quality","volume":"81","author":"Costa","year":"2015","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"482","DOI":"10.1109\/36.142926","article-title":"Multispectral classification of Landsat-images using neural networks","volume":"30","author":"Bischof","year":"1992","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"823","DOI":"10.1080\/01431160600746456","article-title":"A survey of image classification methods and techniques for improving classification performance","volume":"28","author":"Lu","year":"2007","journal-title":"Int. J. Remote Sens."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"850","DOI":"10.1016\/j.envsoft.2008.11.012","article-title":"Increasing the accuracy of neural network classification using refined training data","volume":"24","author":"Kavzoglu","year":"2009","journal-title":"Environ. Model. Softw."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"3549","DOI":"10.1080\/014311699211192","article-title":"The significance of border training patterns in classification by a feedforward neural network using backpropagation learning","volume":"20","author":"Foody","year":"1999","journal-title":"Int. J. Remote Sens."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"247","DOI":"10.1016\/j.isprsjprs.2010.11.001","article-title":"Support vector machines in remote sensing: A review","volume":"66","author":"Mountrakis","year":"2011","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"725","DOI":"10.1080\/01431160110040323","article-title":"An assessment of support vector machines for land cover classification","volume":"23","author":"Huang","year":"2002","journal-title":"Int. J. Remote Sens."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1007","DOI":"10.1080\/01431160512331314083","article-title":"Support vector machines for classification in remote sensing","volume":"26","author":"Pal","year":"2005","journal-title":"Int. J. Remote Sens."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"161","DOI":"10.3390\/rs8020161","article-title":"Tree species abundance predictions in a tropical agricultural landscape with a supervised classification model and imbalanced data","volume":"8","author":"Graves","year":"2016","journal-title":"Remote Sens."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"273","DOI":"10.1007\/BF00994018","article-title":"Support-vector networks","volume":"20","author":"Cortes","year":"1995","journal-title":"Mach. Learn."},{"key":"ref_28","unstructured":"Mercier, G., and Lennon, M. (2003, January 21\u201325). Support vector machines for hyperspectral image classification with spectral-based kernels. Proceedings of the IEEE International Geoscience and Remote Sensing Symposium, Toulouse, France."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1778","DOI":"10.1109\/TGRS.2004.831865","article-title":"Classification of hyperspectral remote sensing images with support vector machines","volume":"42","author":"Melgani","year":"2004","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"373","DOI":"10.1080\/17538947.2012.713190","article-title":"Global characterization and monitoring of forest cover using Landsat data: Opportunities and challenges","volume":"5","author":"Townshend","year":"2012","journal-title":"Int. J. Digit. Earth"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"2297","DOI":"10.1109\/TGRS.2009.2039484","article-title":"Feature selection for classification of hyperspectral data by SVM","volume":"48","author":"Pal","year":"2010","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1016\/j.rse.2004.06.017","article-title":"Toward intelligent training of supervised image classifications: Directing training data acquisition for SVM classification","volume":"93","author":"Foody","year":"2004","journal-title":"Remote Sens. Environ."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"193","DOI":"10.1021\/ci050374h","article-title":"Enrichment of high-throughput screening data with increasing levels of noise using support vector machines, recursive partitioning, and laplacian-modified naive Bayesian classifiers","volume":"46","author":"Meir","year":"2006","journal-title":"J. Chem. Inf. Model."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"101","DOI":"10.1016\/j.neucom.2012.11.023","article-title":"Fuzzy support vector machine based on within-class scatter for classification problems with outliers or noises","volume":"110","author":"An","year":"2013","journal-title":"Neurocomputing"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1344","DOI":"10.1109\/JSTARS.2012.2215310","article-title":"Evaluation of SVM, RVM and SMLR for accurate image classification with limited ground data","volume":"5","author":"Pal","year":"2012","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"787","DOI":"10.1080\/13658810500072020","article-title":"Comparison of land cover maps using fuzzy agreement","volume":"19","author":"Fritz","year":"2005","journal-title":"Int. J. Geogr. Inf. Sci."},{"key":"ref_37","unstructured":"Anderson, R.P., Araujo, M., Guisan, A., Lobo, J.M., Martinez-Meyer, E., Townsend, A., and Soberon, J. (2016). The Case of the Global Biodiversity Information Facility (GBIF), 2016, Global Biodiversity Information Facility. Final Report of the Task Group on GBIF Data Fitness for Use in Distribution Modelling."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"799","DOI":"10.1080\/014311697218764","article-title":"An evaluation of some factors affecting the accuracy of classification by an artificial neural network","volume":"18","author":"Foody","year":"1997","journal-title":"Int. J. Remote Sens."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Mather, P.M., and Koch, M. (2011). Computer Processing of Remotely-Sensed Images: An Introduction, Wiley. [4th ed.].","DOI":"10.1002\/9780470666517"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"627","DOI":"10.14358\/PERS.70.5.627","article-title":"Thematic map comparison: Evaluating the statistical significance of differences in classification accuracy","volume":"70","author":"Foody","year":"2004","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Fleiss, J.L., Levin, B., Paik, M.C., and Fleiss, J. (2003). Statistical Methods for Rates & Proportions, Wiley-Interscience. [3rd ed.].","DOI":"10.1002\/0471445428"},{"key":"ref_42","first-page":"193","article-title":"An automated land use mapping comparison of the Bayesian maximum likelihood and linear discriminant analysis algorithms","volume":"50","author":"Tom","year":"1984","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"1461","DOI":"10.1080\/01431169508954488","article-title":"Components of accuracy of maps with special reference to discriminant analysis of remote sensor data","volume":"16","author":"Lark","year":"1995","journal-title":"Int. J. Remote Sens."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Vapnik, V.N. (1995). The Nature of Statistical Learning Theory, Springer.","DOI":"10.1007\/978-1-4757-2440-0"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Camps-Valls, G., and Bruzzone, L. (2009). Kernel Methods for Remote Sensing Data Analysis, Wiley & Sons.","DOI":"10.1002\/9780470748992"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Foody, G.M. (2015, January 26\u201331). The effect of mis-labeled training data on the accuracy of supervised image classification by SVM. Proceedings of the IEEE Internal Geoscience Remote Sensings Symtem, Milan, Italy.","DOI":"10.1109\/IGARSS.2015.7326952"},{"key":"ref_47","first-page":"211","article-title":"Sparse Bayesian learning and the relevance vector machine","volume":"1","author":"Tipping","year":"2001","journal-title":"J. Mach. Learn. Res."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"586","DOI":"10.1109\/LGRS.2007.903069","article-title":"Hyperspectral image classification using relevance vector machines","volume":"4","author":"Demir","year":"2007","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"1817","DOI":"10.1080\/01431160701822115","article-title":"RVM-based multi-class classification of remotely sensed data","volume":"29","author":"Foody","year":"2008","journal-title":"Int. J. Remote Sens."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"2100","DOI":"10.1109\/TGRS.2010.2103381","article-title":"Robust hyperspectral classification using relevance vector machine","volume":"49","author":"Mianji","year":"2011","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"957","DOI":"10.1109\/TPAMI.2005.127","article-title":"Sparse multinomial logistic regression: Fast algorithms and generalization bounds","volume":"27","author":"Krishnapuram","year":"2005","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Hastie, T., Tibshirani, R., and Friedman, J. (2001). The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Springer-Verlag.","DOI":"10.1007\/978-0-387-21606-5"}],"container-title":["ISPRS International Journal of Geo-Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2220-9964\/5\/11\/199\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T19:34:26Z","timestamp":1760211266000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2220-9964\/5\/11\/199"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,11,1]]},"references-count":52,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2016,11]]}},"alternative-id":["ijgi5110199"],"URL":"https:\/\/doi.org\/10.3390\/ijgi5110199","relation":{},"ISSN":["2220-9964"],"issn-type":[{"value":"2220-9964","type":"electronic"}],"subject":[],"published":{"date-parts":[[2016,11,1]]}}}