{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,2]],"date-time":"2026-06-02T00:42:30Z","timestamp":1780360950361,"version":"3.54.1"},"reference-count":228,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2020,3,23]],"date-time":"2020-03-23T00:00:00Z","timestamp":1584921600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Omidyar Network","award":["PlaceFund"],"award-info":[{"award-number":["PlaceFund"]}]},{"DOI":"10.13039\/100000104","name":"National Aeronautics and Space Administration","doi-asserted-by":"publisher","award":["80NSSC18K0158"],"award-info":[{"award-number":["80NSSC18K0158"]}],"id":[{"id":"10.13039\/100000104","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["SES-1801251"],"award-info":[{"award-number":["SES-1801251"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000161","name":"National Institute of Standards and Technology","doi-asserted-by":"publisher","award":["2017-67003-26615"],"award-info":[{"award-number":["2017-67003-26615"]}],"id":[{"id":"10.13039\/100000161","id-type":"DOI","asserted-by":"publisher"}]},{"name":"National Institute of Standards and Technolgy Undergraduate Research Fellowship Program","award":["-"],"award-info":[{"award-number":["-"]}]},{"DOI":"10.13039\/100004855","name":"New York State Department of Environmental Conservation","doi-asserted-by":"publisher","award":["DEC01-T00640GG-3350000"],"award-info":[{"award-number":["DEC01-T00640GG-3350000"]}],"id":[{"id":"10.13039\/100004855","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Remote sensing, or Earth Observation (EO), is increasingly used to understand Earth system dynamics and create continuous and categorical maps of biophysical properties and land cover, especially based on recent advances in machine learning (ML). ML models typically require large, spatially explicit training datasets to make accurate predictions. Training data (TD) are typically generated by digitizing polygons on high spatial-resolution imagery, by collecting in situ data, or by using pre-existing datasets. TD are often assumed to accurately represent the truth, but in practice almost always have error, stemming from (1) sample design, and (2) sample collection errors. The latter is particularly relevant for image-interpreted TD, an increasingly commonly used method due to its practicality and the increasing training sample size requirements of modern ML algorithms. TD errors can cause substantial errors in the maps created using ML algorithms, which may impact map use and interpretation. Despite these potential errors and their real-world consequences for map-based decisions, TD error is often not accounted for or reported in EO research. Here we review the current practices for collecting and handling TD. We identify the sources of TD error, and illustrate their impacts using several case studies representing different EO applications (infrastructure mapping, global surface flux estimates, and agricultural monitoring), and provide guidelines for minimizing and accounting for TD errors. To harmonize terminology, we distinguish TD from three other classes of data that should be used to create and assess ML models: training reference data, used to assess the quality of TD during data generation; validation data, used to iteratively improve models; and map reference data, used only for final accuracy assessment. We focus primarily on TD, but our advice is generally applicable to all four classes, and we ground our review in established best practices for map accuracy assessment literature. EO researchers should start by determining the tolerable levels of map error and appropriate error metrics. Next, TD error should be minimized during sample design by choosing a representative spatio-temporal collection strategy, by using spatially and temporally relevant imagery and ancillary data sources during TD creation, and by selecting a set of legend definitions supported by the data. Furthermore, TD error can be minimized during the collection of individual samples by using consensus-based collection strategies, by directly comparing interpreted training observations against expert-generated training reference data to derive TD error metrics, and by providing image interpreters with thorough application-specific training. We strongly advise that TD error is incorporated in model outputs, either directly in bias and variance estimates or, at a minimum, by documenting the sources and implications of error. TD should be fully documented and made available via an open TD repository, allowing others to replicate and assess its use. To guide researchers in this process, we propose three tiers of TD error accounting standards. Finally, we advise researchers to clearly communicate the magnitude and impacts of TD error on map outputs, with specific consideration given to the likely map audience.<\/jats:p>","DOI":"10.3390\/rs12061034","type":"journal-article","created":{"date-parts":[[2020,3,24]],"date-time":"2020-03-24T07:16:08Z","timestamp":1585034168000},"page":"1034","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":93,"title":["Accounting for Training Data Error in Machine Learning Applied to Earth Observations"],"prefix":"10.3390","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4568-5758","authenticated-orcid":false,"given":"Arthur","family":"Elmes","sequence":"first","affiliation":[{"name":"Graduate School of Geography, Clark University, Worcester, MA 01610, USA"},{"name":"School for the Environment, University of Massachusetts Boston, Boston, MA 02125, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5662-3643","authenticated-orcid":false,"given":"Hamed","family":"Alemohammad","sequence":"additional","affiliation":[{"name":"Radiant Earth Foundation, San Francisco, CA, 94105, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ryan","family":"Avery","sequence":"additional","affiliation":[{"name":"Department of Geography, University of California, Santa Barbara, CA 93013, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kelly","family":"Caylor","sequence":"additional","affiliation":[{"name":"Department of Geography, University of California, Santa Barbara, CA 93013, USA"},{"name":"Bren School of Environmental Science and Management, University of California, Santa Barbara, CA 93013, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"J.","family":"Eastman","sequence":"additional","affiliation":[{"name":"Graduate School of Geography, Clark University, Worcester, MA 01610, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lewis","family":"Fishgold","sequence":"additional","affiliation":[{"name":"Azavea, Inc., Philadelphia, PA 19123, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mark","family":"Friedl","sequence":"additional","affiliation":[{"name":"Department of Earth and Environment, Boston University, Boston, MA 02215"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Meha","family":"Jain","sequence":"additional","affiliation":[{"name":"School for Environment and Sustainability, University of Michigan, Ann Arbor, MI 48109, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Divyani","family":"Kohli","sequence":"additional","affiliation":[{"name":"Faculty of Geo-Information Science &amp; Earth Observation (ITC), University of Twente, 7514 AE Enschede, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2844-3842","authenticated-orcid":false,"given":"Juan","family":"Laso Bayas","sequence":"additional","affiliation":[{"name":"Center for Earth Observation and Citizen Science, Ecosystems Services and Management Program, International Institute for Applied Systems Analysis (IIASA), Laxenburg A-2361, Austria"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dalton","family":"Lunga","sequence":"additional","affiliation":[{"name":"National Security Emerging Technologies, Oak Ridge National Laboratory, Oak Ridge, TN 37831, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3333-3931","authenticated-orcid":false,"given":"Jessica","family":"McCarty","sequence":"additional","affiliation":[{"name":"Department of Geography and Geospatial Analysis Center, Miami University, Oxford, OH 45056, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7287-5875","authenticated-orcid":false,"given":"Robert","family":"Pontius","sequence":"additional","affiliation":[{"name":"Graduate School of Geography, Clark University, Worcester, MA 01610, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andrew","family":"Reinmann","sequence":"additional","affiliation":[{"name":"Environmental Sciences Initiative, CUNY Advanced Science Research Center, New York, NY 10065, USA"},{"name":"Department of Geography and Environmental Science, Hunter College, New York, NY 10065, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"John","family":"Rogan","sequence":"additional","affiliation":[{"name":"Graduate School of Geography, Clark University, Worcester, MA 01610, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lei","family":"Song","sequence":"additional","affiliation":[{"name":"Graduate School of Geography, Clark University, Worcester, MA 01610, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hristiana","family":"Stoynova","sequence":"additional","affiliation":[{"name":"Environmental Sciences Initiative, CUNY Advanced Science Research Center, New York, NY 10065, USA"},{"name":"Department of Geography and Environmental Science, Hunter College, New York, NY 10065, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Su","family":"Ye","sequence":"additional","affiliation":[{"name":"Graduate School of Geography, Clark University, Worcester, MA 01610, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhuang-Fang","family":"Yi","sequence":"additional","affiliation":[{"name":"Development Seed, Washington, DC 20001, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9358-816X","authenticated-orcid":false,"given":"Lyndon","family":"Estes","sequence":"additional","affiliation":[{"name":"Graduate School of Geography, Clark University, Worcester, MA 01610, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2020,3,23]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1016\/j.isprsjprs.2014.09.002","article-title":"Global Land Cover Mapping at 30 m Resolution: A POK-Based Operational Approach","volume":"103","author":"Chen","year":"2015","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"168","DOI":"10.1016\/j.rse.2009.08.016","article-title":"MODIS Collection 5 global land cover: Algorithm refinements and characterization of new datasets","volume":"114","author":"Friedl","year":"2010","journal-title":"Remote Sens. Environ."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"639","DOI":"10.1038\/s41586-018-0411-9","article-title":"Global land change from 1982 to 2016","volume":"560","author":"Song","year":"2018","journal-title":"Nature"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Mohanty, B.P., Cosh, M.H., Lakshmi, V., and Montzka, C. (2017). Soil Moisture Remote Sensing: State-of-the-Science. Vadose Zone J., 16.","DOI":"10.2136\/vzj2016.10.0105"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Daudt, R.C., Le Saux, B., Boulch, A., and Gousseau, Y. (2019). Guided Anisotropic Diffusion and Iterative Learning for Weakly Supervised Change Detection. arXiv.","DOI":"10.1109\/CVPRW.2019.00187"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1080\/23729333.2015.1055644","article-title":"Automatic identification of building types based on topographic databases\u2014A comparison of different data sources","volume":"1","author":"Hecht","year":"2015","journal-title":"Int. J. Cartogr."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1016\/j.agrformet.2018.03.003","article-title":"Evaluation of land surface phenology from VIIRS data using time series of PhenoCam imagery","volume":"256\u2013257","author":"Zhang","year":"2018","journal-title":"Agric. For. Meteorol."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"361","DOI":"10.1109\/JSTARS.2010.2075916","article-title":"An Enhanced TIMESAT Algorithm for Estimating Vegetation Phenology Metrics From MODIS Data","volume":"4","author":"Tan","year":"2011","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"981","DOI":"10.1029\/2006JG000217","article-title":"Global vegetation phenology from Moderate Resolution Imaging Spectroradiometer (MODIS): Evaluation of global patterns and comparison with in situ measurements: GLOBAL PHENOLOGY FROM MODIS","volume":"111","author":"Zhang","year":"2006","journal-title":"J. Geophys. Res."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1016\/S0034-4257(02)00091-3","article-title":"First operational BRDF, albedo nadir reflectance products from MODIS","volume":"83","author":"Schaaf","year":"2002","journal-title":"Remote Sens. Environ."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"256","DOI":"10.1016\/j.rse.2017.09.020","article-title":"Evaluation of the VIIRS BRDF, Albedo and NBAR products suite and an assessment of continuity with the long term MODIS record","volume":"201","author":"Liu","year":"2017","journal-title":"Remote Sens. Environ."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1016\/j.rse.2018.02.001","article-title":"Capturing rapid land surface dynamics with Collection V006 MODIS BRDF\/NBAR\/Albedo (MCD43) products","volume":"207","author":"Wang","year":"2018","journal-title":"Remote Sens. Environ."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1016\/j.rse.2006.06.026","article-title":"New refinements and validation of the MODIS Land-Surface Temperature\/Emissivity products","volume":"112","author":"Wan","year":"2008","journal-title":"Remote Sens. Environ."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1840","DOI":"10.1109\/LGRS.2014.2312032","article-title":"Land Surface Temperature Retrieval Methods From Landsat-8 Thermal Infrared Sensor Data","volume":"11","author":"Sobrino","year":"2014","journal-title":"IEEE Geoscie. Remote Sens. Lett."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"790","DOI":"10.1126\/science.aaf7894","article-title":"Combining satellite imagery and machine learning to predict poverty","volume":"353","author":"Jean","year":"2016","journal-title":"Science"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"418","DOI":"10.1038\/nature20584","article-title":"High-resolution mapping of global surface water and its long-term changes","volume":"540","author":"Pekel","year":"2016","journal-title":"Nature"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Hansen, M.C., Potapov, P., and Tyukavina, A. (2019). Comment on \u201cTropical forests are a net carbon source based on aboveground measurements of gain and loss\u201d. Science, 363.","DOI":"10.1126\/science.aar3629"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"325","DOI":"10.1017\/S0376892912000173","article-title":"Influence of carbon mapping and land change modelling on the prediction of carbon emissions from deforestation","volume":"39","author":"Pontius","year":"2012","journal-title":"Environ. Conserv."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. (2009, January 20\u201325). Imagenet: A large-scale hierarchical image database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Helber, P., Bischke, B., Dengel, A., and Borth, D. (2019). EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens, 1\u201310.","DOI":"10.1109\/IGARSS.2018.8519248"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"117","DOI":"10.1109\/TGRS.2017.2743243","article-title":"Learning Multiscale Deep Features for High-Resolution Satellite Image Scene Classification","volume":"56","author":"Liu","year":"2018","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"170136","DOI":"10.1038\/sdata.2017.136","article-title":"A global reference database of crowdsourced cropland data collected using the Geo-Wiki platform","volume":"4","author":"Lesiv","year":"2017","journal-title":"Sci. Data"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Mathieu, P.-P., and Aubrecht, C. (2018). Machine Learning Applications for Earth Observation. Earth Observation Open Science and Innovation, Springer International Publishing.","DOI":"10.1007\/978-3-319-65633-5"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1016\/j.gsf.2015.07.003","article-title":"Machine learning in geosciences and remote sensing","volume":"7","author":"Lary","year":"2016","journal-title":"Geosci. Front."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1303","DOI":"10.1080\/014311600210191","article-title":"Development of a global land cover characteristics database and IGBP DISCover from 1 km AVHRR data","volume":"21","author":"Loveland","year":"2000","journal-title":"Int. J. Remote Sens."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1016\/j.rse.2018.12.013","article-title":"Hierarchical mapping of annual global land cover 2001 to present: The MODIS Collection 6 Land Cover product","volume":"222","author":"Gray","year":"2019","journal-title":"Remote Sens. Environ."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"181","DOI":"10.14358\/PERS.77.2.181","article-title":"Utilizing Temporally Invariant Calibration Sites to Classify Multiple Dates and Types of Satellite Imagery","volume":"77","author":"Fortier","year":"2011","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1016\/j.rse.2004.06.017","article-title":"Toward intelligent training of supervised image classifications: Directing training data acquisition for SVM classification","volume":"93","author":"Foody","year":"2004","journal-title":"Remote Sens. Environ."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Graves, S.J., Asner, G.P., Martin, R.E., Anderson, C.B., Colgan, M.S., Kalantari, L., and Bohlman, S.A. (2016). Tree Species Abundance Predictions in a Tropical Agricultural Landscape with a Supervised Classification Model and Imbalanced Data. Remote Sens., 8.","DOI":"10.3390\/rs8020161"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Foody, G., Pal, M., Rocchini, D., and Garzon-Lopez, C. (2016). The sensitivity of mapping methods to reference data quality: Training supervised image classifications with imperfect reference data. Int. J. Geo-Inf., 5.","DOI":"10.3390\/ijgi5110199"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"2784","DOI":"10.1080\/01431161.2018.1433343","article-title":"Implementation of machine-learning classification in remote sensing: an applied review","volume":"39","author":"Maxwell","year":"2018","journal-title":"Int. J. Remote Sens."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"725","DOI":"10.1080\/01431160110040323","article-title":"An assessment of support vector machines for land cover classification","volume":"23","author":"Huang","year":"2002","journal-title":"Int. J. Remote Sens."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"322","DOI":"10.1111\/gcb.13904","article-title":"A large-area, spatially continuous assessment of land cover map error and its impact on downstream analyses","volume":"24","author":"Estes","year":"2018","journal-title":"Glob. Chang. Biol."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Pengra, B.W., Stehman, S.V., Horton, J.A., Dockter, D.J., Schroeder, T.A., Yang, Z., Cohen, W.B., Healey, S.P., and Loveland, T.R. (2019). Quality control and assessment of interpreter consistency of annual land cover reference data in an operational national monitoring program. Remote Sens. Environ., 111261.","DOI":"10.1016\/j.rse.2019.111261"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1109\/MGRS.2017.2762307","article-title":"Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources","volume":"5","author":"Zhu","year":"2017","journal-title":"IEEE Geosci. Remote Sens. Mag."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"166","DOI":"10.1016\/j.isprsjprs.2019.04.015","article-title":"Deep learning in remote sensing applications: A meta-analysis and review","volume":"152","author":"Ma","year":"2019","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1016\/S0034-4257(01)00295-4","article-title":"Status of land cover classification accuracy assessment","volume":"80","author":"Foody","year":"2002","journal-title":"Remote Sens. Environ."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"2271","DOI":"10.1016\/j.rse.2010.05.003","article-title":"Assessing the accuracy of land cover change with imperfect ground reference data","volume":"114","author":"Foody","year":"2010","journal-title":"Remote Sens. Environ."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"42","DOI":"10.1016\/j.rse.2014.02.015","article-title":"Good practices for estimating area and assessing accuracy of land change","volume":"148","author":"Olofsson","year":"2014","journal-title":"Remote Sens. Environ."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"4407","DOI":"10.1080\/01431161.2011.552923","article-title":"Death to Kappa: Birth of quantity disagreement and allocation disagreement for accuracy assessment","volume":"32","author":"Pontius","year":"2011","journal-title":"Int. J. Remote Sens."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Congalton, R.G., and Green, K. (2008). Assessing the Accuracy of Remotely Sensed Data: Principles and Practices, CRC Press.","DOI":"10.1201\/9781420055139"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"373","DOI":"10.1191\/0309132506ph612pr","article-title":"Cartography: Uncertainty, interventions, and dynamic display","volume":"30","author":"Monmonier","year":"2006","journal-title":"Prog. Hum. Geogr."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"10","DOI":"10.14714\/CP13.1000","article-title":"Visualizing Uncertain Information","volume":"1","author":"MacEachren","year":"1992","journal-title":"Cartogr. Perspect."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Goodchild, M.F., and Gopal, S. (1989). The Accuracy of Spatial Databases, CRC Press.","DOI":"10.1201\/b12612"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1016\/0034-4257(91)90048-B","article-title":"A review of assessing the accuracy of classifications of remotely sensed data","volume":"37","author":"Congalton","year":"1991","journal-title":"Remote Sens. Environ."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"292","DOI":"10.1016\/j.isprsjprs.2018.06.002","article-title":"The effects of imperfect reference data on remote sensing-assisted estimators of land cover class proportions","volume":"142","author":"McRoberts","year":"2018","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"4831","DOI":"10.1080\/01431160802672864","article-title":"Effect of errors in ground truth on classification accuracy","volume":"30","author":"Carlotto","year":"2009","journal-title":"Int. J. Remote Sens."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"155","DOI":"10.1016\/j.isprsjprs.2015.03.014","article-title":"Exploring issues of training data imbalance and mislabelling on random forest performance for large area land cover classification using the ensemble margin","volume":"105","author":"Mellor","year":"2015","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Swan, B., Laverdiere, M., and Yang, H.L. (2018, January 6). How Good is Good Enough?: Quantifying the Effects of Training Set Quality. Proceedings of the 2Nd ACM SIGSPATIAL International Workshop on AI for Geographic Knowledge Discovery, Seattle, WA, USA.","DOI":"10.1145\/3281548.3281557"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"623","DOI":"10.2747\/1548-1603.49.5.623","article-title":"An Evaluation of Bagging, Boosting, and Random Forests for Land-Cover Classification in Cape Cod, Massachusetts, USA","volume":"49","author":"Ghimire","year":"2012","journal-title":"GISci. Remote Sens."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1016\/j.isprsjprs.2011.11.002","article-title":"An assessment of the effectiveness of a random forest classifier for land-cover classification","volume":"67","author":"Ghimire","year":"2012","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"2142","DOI":"10.1109\/TGRS.2008.2011983","article-title":"A Novel Context-Sensitive Semisupervised SVM Classifier Robust to Mislabeled Training Samples","volume":"47","author":"Bruzzone","year":"2009","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"22","DOI":"10.1016\/j.cageo.2013.10.008","article-title":"Geological mapping using remote sensing data: A comparison of five machine learning algorithms, their response to variations in the spatial distribution of training data and the use of explicit spatial information","volume":"63","author":"Cracknell","year":"2014","journal-title":"Comput. Geosci."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1016\/j.isprsjprs.2017.04.017","article-title":"Exploring diversity in ensemble classification: Applications in large area land cover mapping","volume":"129","author":"Mellor","year":"2017","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Xiong, J., Thenkabail, P.S., Tilton, J.C., Gumma, M.K., Teluguntla, P., Oliphant, A., Congalton, R.G., Yadav, K., and Gorelick, N. (2017). Nominal 30-m Cropland Extent Map of Continental Africa by Integrating Pixel-Based and Object-Based Algorithms Using Sentinel-2 and Landsat-8 Data on Google Earth Engine. Remote Sens., 9.","DOI":"10.3390\/rs9101065"},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"111611","DOI":"10.1016\/j.rse.2019.111611","article-title":"Mapping smallholder and large-scale cropland dynamics with a flexible classification system and pixel-based composites in an emerging frontier of Mozambique","volume":"239","author":"Bey","year":"2020","journal-title":"Remote Sens. Environ."},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"111199","DOI":"10.1016\/j.rse.2019.05.018","article-title":"Key issues in rigorous accuracy assessment of land cover products","volume":"231","author":"Stehman","year":"2019","journal-title":"Remote Sens. Environ."},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"233","DOI":"10.1007\/s13157-012-0373-x","article-title":"Object-based Vegetation Mapping in the Kissimmee River Watershed Using HyMap Data and Machine Learning Techniques","volume":"33","author":"Zhang","year":"2013","journal-title":"Wetlands"},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"2272","DOI":"10.1016\/j.rse.2007.10.004","article-title":"Mapping land-cover modifications over large areas: A comparison of machine learning algorithms","volume":"112","author":"Rogan","year":"2008","journal-title":"Remote Sens. Environ."},{"key":"ref_60","doi-asserted-by":"crossref","unstructured":"Copass, C., Antonova, N., and Kennedy, R. (2018). Comparison of Office and Field Techniques for Validating Landscape Change Classification in Pacific Northwest National Parks. Remote Sens., 11.","DOI":"10.3390\/rs11010003"},{"key":"ref_61","doi-asserted-by":"crossref","unstructured":"Lesiv, M., See, L., Laso Bayas, J.C., Sturn, T., Schepaschenko, D., Karner, M., Moorthy, I., McCallum, I., and Fritz, S. (2018). Characterizing the Spatial and Temporal Availability of Very High Resolution Satellite Imagery in Google Earth and Microsoft Bing Maps as a Source of Reference Data. Land, 7.","DOI":"10.5194\/essd-2018-13"},{"key":"ref_62","first-page":"114","article-title":"A global map of rainfed cropland areas (GMRCA) at the end of last millennium using remote sensing","volume":"11","author":"Biradar","year":"2009","journal-title":"Int. J. Appl. Earth Obs. Geoinf."},{"key":"ref_63","doi-asserted-by":"crossref","first-page":"312","DOI":"10.1016\/j.apgeog.2010.07.002","article-title":"Mapping and interpreting historical land cover\/land use changes in a Natura 2000 site using earth observational data: The case of Nestos delta, Greece","volume":"31","author":"Mallinis","year":"2011","journal-title":"Appl. Geogr."},{"key":"ref_64","first-page":"073573","article-title":"Improved land cover mapping using high resolution multiangle 8-band WorldView-2 satellite remote sensing data","volume":"7","author":"Jawak","year":"2013","journal-title":"JARS"},{"key":"ref_65","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1016\/j.isprsjprs.2018.04.002","article-title":"A review of accuracy assessment for object-based image analysis: From per-pixel to per-polygon approaches","volume":"141","author":"Ye","year":"2018","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_66","doi-asserted-by":"crossref","first-page":"170075","DOI":"10.1038\/sdata.2017.75","article-title":"A global dataset of crowdsourced land cover and land use reference data","volume":"4","author":"Fritz","year":"2017","journal-title":"Sci. Data"},{"key":"ref_67","doi-asserted-by":"crossref","first-page":"5243","DOI":"10.1080\/01431160903131000","article-title":"Sampling designs for accuracy assessment of land cover","volume":"30","author":"Stehman","year":"2009","journal-title":"Int. J. Remote Sens."},{"key":"ref_68","doi-asserted-by":"crossref","first-page":"734","DOI":"10.1016\/j.tree.2019.03.006","article-title":"Uncovering Ecological Patterns with Convolutional Neural Networks","volume":"34","author":"Brodrick","year":"2019","journal-title":"Trends Ecol. Evol."},{"key":"ref_69","unstructured":"Xiao, T., Xia, T., Yang, Y., Huang, C., and Wang, X. (2015, January 7\u201312). Learning from massive noisy labeled data for image classification. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA."},{"key":"ref_70","doi-asserted-by":"crossref","first-page":"845","DOI":"10.1109\/TNNLS.2013.2292894","article-title":"Classification in the presence of label noise: A survey","volume":"25","author":"Verleysen","year":"2014","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_71","doi-asserted-by":"crossref","first-page":"131","DOI":"10.1613\/jair.606","article-title":"Identifying Mislabeled Training Data","volume":"11","author":"Brodley","year":"1999","journal-title":"J. Artif. Intell. Res."},{"key":"ref_72","unstructured":"Van Etten, A., Lindenbaum, D., and Bacastow, T.M. (2018). SpaceNet: A Remote Sensing Dataset and Challenge Series. arXiv."},{"key":"ref_73","doi-asserted-by":"crossref","unstructured":"Sumbul, G., Charfuelan, M., Demir, B., and Markl, V. (2019). BigEarthNet: A Large-Scale Benchmark Archive For Remote Sensing Image Understanding. arXiv.","DOI":"10.1109\/IGARSS.2019.8900532"},{"key":"ref_74","doi-asserted-by":"crossref","first-page":"174","DOI":"10.1111\/gcb.14492","article-title":"Estimating the global distribution of field size using crowdsourcing","volume":"25","author":"Lesiv","year":"2019","journal-title":"Glob. Chang. Biol."},{"key":"ref_75","doi-asserted-by":"crossref","first-page":"110","DOI":"10.1016\/j.envsoft.2011.11.015","article-title":"Geo-Wiki: An Online Platform for Improving Global Land Cover","volume":"31","author":"Fritz","year":"2012","journal-title":"Environ. Model. Softw."},{"key":"ref_76","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/s10708-007-9111-y","article-title":"Citizens as sensors: The world of volunteered geography","volume":"69","author":"Goodchild","year":"2007","journal-title":"GeoJournal"},{"key":"ref_77","unstructured":"Kohavi, R. (1995, January 20\u201325). A study of cross-validation and bootstrap for accuracy estimation and model selection. Proceedings of the IJCAI, Montreal, QC, Canada."},{"key":"ref_78","doi-asserted-by":"crossref","first-page":"122","DOI":"10.1016\/j.rse.2012.10.031","article-title":"Making better use of accuracy data in land change studies: Estimating accuracy and area and quantifying uncertainty using stratified estimation","volume":"129","author":"Olofsson","year":"2013","journal-title":"Remote Sens. Environ."},{"key":"ref_79","first-page":"193","article-title":"Performance evaluation metrics for software fault prediction studies","volume":"9","author":"Catal","year":"2012","journal-title":"Acta Polytech. Hung."},{"key":"ref_80","doi-asserted-by":"crossref","unstructured":"Jeni, L.A., Cohn, J.F., and De La Torre, F. (2013, January 2\u20135). Facing Imbalanced Data--Recommendations for the Use of Performance Metrics. Proceedings of the 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction, Geneva, Switzerland.","DOI":"10.1109\/ACII.2013.47"},{"key":"ref_81","doi-asserted-by":"crossref","first-page":"249","DOI":"10.2747\/1548-1603.45.3.249","article-title":"Importance of Matrix Construction for Multiple-Resolution Categorical Map Comparison","volume":"45","author":"Kuzera","year":"2008","journal-title":"GISci. Remote Sens."},{"key":"ref_82","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1007\/s10651-007-0043-y","article-title":"Components of information for multiple resolution comparison between maps that share a real variable","volume":"15","author":"Pontius","year":"2008","journal-title":"Environ. Ecol. Stat."},{"key":"ref_83","doi-asserted-by":"crossref","first-page":"367","DOI":"10.1007\/s10980-013-9984-8","article-title":"Recommendations for using the relative operating characteristic (ROC)","volume":"29","author":"Pontius","year":"2014","journal-title":"Landsc. Ecol."},{"key":"ref_84","first-page":"94","article-title":"Component intensities to relate difference by category with difference overall","volume":"77","author":"Pontius","year":"2019","journal-title":"Int. J. Appl. Earth Obs. Geoinf."},{"key":"ref_85","doi-asserted-by":"crossref","first-page":"963","DOI":"10.14358\/PERS.75.8.963","article-title":"Range of Categorical Associations for Comparison of Maps with Mixed Pixels","volume":"75","author":"Pontius","year":"2009","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_86","doi-asserted-by":"crossref","first-page":"103","DOI":"10.1016\/j.landurbplan.2012.02.010","article-title":"Intensity analysis to unify measurements of size and stationarity of land changes by interval, category, and transition","volume":"106","author":"Aldwaik","year":"2012","journal-title":"Landsc. Urban Plan."},{"key":"ref_87","doi-asserted-by":"crossref","first-page":"351","DOI":"10.3390\/land2030351","article-title":"Design and Interpretation of Intensity Analysis Illustrated by Land Change in Central Kalimantan, Indonesia","volume":"2","author":"Pontius","year":"2013","journal-title":"Land"},{"key":"ref_88","doi-asserted-by":"crossref","first-page":"3137","DOI":"10.1080\/01431160701442120","article-title":"Harshness in image classification accuracy assessment","volume":"29","author":"Foody","year":"2008","journal-title":"Int. J. Remote Sens."},{"key":"ref_89","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1177\/001316446002000104","article-title":"A Coefficient of Agreement for Nominal Scales","volume":"20","author":"Cohen","year":"1960","journal-title":"Educ. Psychol. Meas."},{"key":"ref_90","doi-asserted-by":"crossref","first-page":"1223","DOI":"10.1111\/j.1365-2664.2006.01214.x","article-title":"Assessing the Accuracy of Species Distribution Models: Prevalence, Kappa and the True Skill Statistic (TSS)","volume":"43","author":"Allouche","year":"2006","journal-title":"J. Appl. Ecol."},{"key":"ref_91","doi-asserted-by":"crossref","first-page":"111630","DOI":"10.1016\/j.rse.2019.111630","article-title":"Explaining the unsuitability of the kappa coefficient in the assessment and comparison of the accuracy of thematic maps obtained by image classification","volume":"239","author":"Foody","year":"2020","journal-title":"Remote Sens. Environ."},{"key":"ref_92","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1080\/13658810500286976","article-title":"On the use of dimensioned measures of error to evaluate the performance of spatial interpolators","volume":"20","author":"Willmott","year":"2006","journal-title":"Int. J. Geogr. Inf. Sci."},{"key":"ref_93","doi-asserted-by":"crossref","first-page":"749","DOI":"10.1016\/j.atmosenv.2008.10.005","article-title":"Ambiguities inherent in sums-of-squares-based error statistics","volume":"43","author":"Willmott","year":"2009","journal-title":"Atmos. Environ."},{"key":"ref_94","doi-asserted-by":"crossref","first-page":"79","DOI":"10.3354\/cr030079","article-title":"Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance","volume":"30","author":"Willmott","year":"2005","journal-title":"Clim. Res."},{"key":"ref_95","doi-asserted-by":"crossref","first-page":"570","DOI":"10.1080\/13658816.2013.862623","article-title":"The total operating characteristic to measure diagnostic ability for multiple thresholds","volume":"28","author":"Pontius","year":"2014","journal-title":"Int. J. Geogr. Inf. Sci."},{"key":"ref_96","doi-asserted-by":"crossref","first-page":"38","DOI":"10.1017\/S0376892997000088","article-title":"A review of methods for the assessment of prediction errors in conservation presence\/absence models","volume":"24","author":"Fielding","year":"1997","journal-title":"Environ. Conserv."},{"key":"ref_97","doi-asserted-by":"crossref","first-page":"2","DOI":"10.1016\/j.isprsjprs.2009.06.004","article-title":"Object based image analysis for remote sensing","volume":"65","author":"Blaschke","year":"2010","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_98","doi-asserted-by":"crossref","first-page":"338","DOI":"10.1016\/j.rse.2017.11.024","article-title":"Supervised methods of image segmentation accuracy assessment in land cover mapping","volume":"205","author":"Costa","year":"2018","journal-title":"Remote Sens. Environ."},{"key":"ref_99","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1016\/j.rse.2003.12.007","article-title":"Sources of error in accuracy assessment of thematic land-cover maps in the Brazilian Amazon","volume":"90","author":"Powell","year":"2004","journal-title":"Remote Sens. Environ."},{"key":"ref_100","doi-asserted-by":"crossref","first-page":"1790","DOI":"10.1007\/s11430-014-4877-5","article-title":"Land cover mapping using time series HJ-1\/CCD data","volume":"57","author":"Zhong","year":"2014","journal-title":"Sci. China Earth Sci."},{"key":"ref_101","doi-asserted-by":"crossref","first-page":"1276","DOI":"10.1016\/j.rse.2009.02.014","article-title":"A neural network approach using multi-scale textural metrics from very high-resolution panchromatic imagery for urban land-use classification","volume":"113","author":"Pacifici","year":"2009","journal-title":"Remote Sens. Environ."},{"key":"ref_102","first-page":"6","article-title":"Mapping land use-land cover and change detection in Kafur local government, Katsina, Nigeria (1995-2008) using remote sensing and GIS","volume":"2","author":"Abbas","year":"2010","journal-title":"Res. J. Environ. Earth Sci."},{"key":"ref_103","doi-asserted-by":"crossref","first-page":"113","DOI":"10.1007\/s10661-009-0988-4","article-title":"Land cover mapping of the tropical savanna region in Brazil","volume":"166","author":"Sano","year":"2010","journal-title":"Environ. Monit. Assess."},{"key":"ref_104","doi-asserted-by":"crossref","unstructured":"Hu, T., Yang, J., Li, X., and Gong, P. (2016). Mapping Urban Land Use by Using Landsat Images and Open Social Data. Remote Sens., 8.","DOI":"10.3390\/rs8020151"},{"key":"ref_105","doi-asserted-by":"crossref","first-page":"6089","DOI":"10.3390\/rs6076089","article-title":"Land-Use Mapping in a Mixed Urban-Agricultural Arid Landscape Using Object-Based Image Analysis: A Case Study from Maricopa, Arizona","volume":"6","author":"Galletti","year":"2014","journal-title":"Remote Sens."},{"key":"ref_106","doi-asserted-by":"crossref","first-page":"6026","DOI":"10.3390\/rs5116026","article-title":"Exploring the Use of Google Earth Imagery and Object-Based Methods in Land Use\/Cover Mapping","volume":"5","author":"Hu","year":"2013","journal-title":"Remote Sens."},{"key":"ref_107","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1659\/mrd.0863","article-title":"Incorporating Land Use Mapping and Participation in Jordan: An Approach to Sustainable Management of Two Mountainous Areas","volume":"28","author":"Ajlouni","year":"2008","journal-title":"Mt. Res. Dev."},{"key":"ref_108","doi-asserted-by":"crossref","first-page":"195","DOI":"10.1007\/s11442-014-1082-6","article-title":"Spatiotemporal characteristics, patterns, and causes of land-use changes in China since the late 1980s","volume":"24","author":"Liu","year":"2014","journal-title":"J. Geogr. Sci."},{"key":"ref_109","unstructured":"Yadav, P.K., Kapoor, M., and Sarma, K. (2012). Land Use Land Cover Mapping, Change Detection and Conflict Analysis of Nagzira-Navegaon Corridor, Central India Using Geospatial Technology. Int. J. Remote Sens. GIS, 1."},{"key":"ref_110","doi-asserted-by":"crossref","first-page":"2956","DOI":"10.1109\/TGRS.2008.2000630","article-title":"Land Use and Land Cover Mapping in the Brazilian Amazon Using Polarimetric Airborne P-Band SAR Data","volume":"46","author":"Dutra","year":"2008","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_111","doi-asserted-by":"crossref","first-page":"390","DOI":"10.1016\/j.apgeog.2008.12.005","article-title":"Land use and land cover change in Greater Dhaka, Bangladesh: Using remote sensing to promote sustainable urbanization","volume":"29","author":"Dewan","year":"2009","journal-title":"Appl. Geogr."},{"key":"ref_112","doi-asserted-by":"crossref","first-page":"2270","DOI":"10.1016\/j.jenvman.2007.06.030","article-title":"Land cover mapping of wetland areas in an agricultural landscape using SAR and Landsat imagery","volume":"90","author":"Ducrot","year":"2009","journal-title":"J. Environ. Manag."},{"key":"ref_113","doi-asserted-by":"crossref","first-page":"2088","DOI":"10.1109\/JSTARS.2012.2228167","article-title":"A Pixel-Based Landsat Compositing Algorithm for Large Area Land Cover Mapping","volume":"6","author":"Griffiths","year":"2013","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_114","first-page":"115","article-title":"Sub-pixel land-cover mapping with improved fraction images upon multiple-point simulation","volume":"22","author":"Ge","year":"2013","journal-title":"Int. J. Appl. Earth Obs. Geoinf."},{"key":"ref_115","doi-asserted-by":"crossref","first-page":"2607","DOI":"10.1080\/01431161.2012.748992","article-title":"Finer resolution observation and monitoring of global land cover: First mapping results with Landsat TM and ETM+ data","volume":"34","author":"Gong","year":"2013","journal-title":"Int. J. Remote Sens."},{"key":"ref_116","first-page":"576","article-title":"Land use mapping using visual vs. digital image interpretation of TM and Google earth derived imagery in Shrivan-Darasi watershed (Northwest of Iran)","volume":"3","author":"Ghorbani","year":"2013","journal-title":"Eur. J. Exp. Biol."},{"key":"ref_117","doi-asserted-by":"crossref","first-page":"187","DOI":"10.1016\/j.landurbplan.2009.05.001","article-title":"Spatio-temporal dynamics and evolution of land use change and landscape pattern in response to rapid urbanization","volume":"92","author":"Deng","year":"2009","journal-title":"Landsc. Urban Plan."},{"key":"ref_118","first-page":"S27","article-title":"Land cover change assessment using decision trees, support vector machines and maximum likelihood classification algorithms","volume":"12","author":"Otukei","year":"2010","journal-title":"Int. J. Appl. Earth Obs. Geoinf."},{"key":"ref_119","doi-asserted-by":"crossref","first-page":"1025","DOI":"10.1080\/13658816.2011.566569","article-title":"Hybrid object-based approach for land use\/land cover mapping using high spatial resolution imagery","volume":"25","author":"Malinverni","year":"2011","journal-title":"Int. J. Geogr. Inf. Sci."},{"key":"ref_120","doi-asserted-by":"crossref","first-page":"533","DOI":"10.1016\/j.apgeog.2010.11.006","article-title":"Comparison of methods for land-use classification incorporating remote sensing and GIS inputs","volume":"31","author":"Rozenstein","year":"2011","journal-title":"Appl. Geogr."},{"key":"ref_121","doi-asserted-by":"crossref","first-page":"169","DOI":"10.1080\/13658816.2011.577745","article-title":"Large-scale land cover mapping with the integration of multi-source information based on the Dempster\u2013Shafer theory","volume":"26","author":"Ran","year":"2012","journal-title":"Int. J. Geogr. Inf. Sci."},{"key":"ref_122","doi-asserted-by":"crossref","first-page":"2816","DOI":"10.1016\/j.rse.2010.07.001","article-title":"A scalable approach to mapping annual land cover at 250 m using MODIS time series data: A case study in the Dry Chaco ecoregion of South America","volume":"114","author":"Clark","year":"2010","journal-title":"Remote Sens. Environ."},{"key":"ref_123","first-page":"46","article-title":"Assessing different remote sensing techniques to detect land use\/cover changes in the eastern Mediterranean","volume":"11","author":"Berberoglu","year":"2009","journal-title":"Int. J. Appl. Earth Obs. Geoinf."},{"key":"ref_124","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random Forests","volume":"45","author":"Breiman","year":"2001","journal-title":"Mach. Learn."},{"key":"ref_125","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.ecolmodel.2012.03.007","article-title":"Evaluating effectiveness of down-sampling for stratified designs and unbalanced prevalence in Random Forest models of tree species distributions in Nevada","volume":"233","author":"Freeman","year":"2012","journal-title":"Ecol. Modell."},{"key":"ref_126","doi-asserted-by":"crossref","first-page":"373","DOI":"10.1080\/17538947.2012.713190","article-title":"Global characterization and monitoring of forest cover using Landsat data: Opportunities and challenges","volume":"5","author":"Townshend","year":"2012","journal-title":"Int. J. Digit. Earth"},{"key":"ref_127","doi-asserted-by":"crossref","first-page":"78","DOI":"10.1016\/j.isprsjprs.2012.04.001","article-title":"Comparison of support vector machine, neural network, and CART algorithms for the land-cover classification using limited training data points","volume":"70","author":"Shao","year":"2012","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_128","unstructured":"(2017). Planet Team Planet Application Program Interface. Space for Life on Earth, Planet Labs, Inc."},{"key":"ref_129","doi-asserted-by":"crossref","unstructured":"Manfreda, S., McCabe, M.F., Miller, P.E., Lucas, R., Pajuelo Madrigal, V., Mallinis, G., Ben Dor, E., Helman, D., Estes, L., and Ciraolo, G. (2018). On the Use of Unmanned Aerial Systems for Environmental Monitoring. Remote Sens., 10.","DOI":"10.20944\/preprints201803.0097.v1"},{"key":"ref_130","doi-asserted-by":"crossref","unstructured":"Toutin, T. (2001, January 19\u201321). Geometric processing of IKONOS Geo images with DEM. Proceedings of the ISPRS Joint Workshop \u201cHigh Resolution Mapping from Space\u201d 2001, Hanover, Germany.","DOI":"10.4095\/219815"},{"key":"ref_131","doi-asserted-by":"crossref","first-page":"124","DOI":"10.1016\/j.isprsjprs.2010.10.003","article-title":"Orthorectification of VHR optical satellite data exploiting the geometric accuracy of TerraSAR-X data","volume":"66","author":"Reinartz","year":"2011","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_132","first-page":"427","article-title":"Assessing geometric accuracy of the orthorectification process from GeoEye-1 and WorldView-2 panchromatic images","volume":"21","author":"Aguilar","year":"2013","journal-title":"Int. J. Appl. Earth Obs. Geoinf."},{"key":"ref_133","doi-asserted-by":"crossref","unstructured":"Chen, J., and Zipf, A. (2017). DeepVGI: Deep Learning with Volunteered Geographic Information. Proceedings of the Proceedings of the 26th International Conference on World Wide Web Companion, International World Wide Web Conferences Steering Committee.","DOI":"10.1145\/3041021.3054250"},{"key":"ref_134","doi-asserted-by":"crossref","first-page":"6054","DOI":"10.1109\/TGRS.2017.2719738","article-title":"Learning Aerial Image Segmentation From Online Maps","volume":"55","author":"Kaiser","year":"2017","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_135","doi-asserted-by":"crossref","unstructured":"Audebert, N., Le Saux, B., and Lef\u00e8vre, S. (2017, January 21\u201326). Joint learning from earth observation and openstreetmap data to get faster better semantic maps. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, HI, USA.","DOI":"10.1109\/CVPRW.2017.199"},{"key":"ref_136","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1016\/0034-4257(86)90018-0","article-title":"On the nature of models in remote sensing","volume":"20","author":"Strahler","year":"1986","journal-title":"Remote Sens. Environ."},{"key":"ref_137","first-page":"491","article-title":"Relating the land-cover composition of mixed pixels to artificial neural network classification output","volume":"62","author":"Foody","year":"1996","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_138","doi-asserted-by":"crossref","first-page":"329","DOI":"10.1016\/S0034-4257(96)00107-1","article-title":"Artificial neural network response to mixed pixels in coarse-resolution satellite data","volume":"58","author":"Moody","year":"1996","journal-title":"Remote Sens. Environ."},{"key":"ref_139","doi-asserted-by":"crossref","first-page":"3141","DOI":"10.1080\/014311698214235","article-title":"Global land cover classifications at 8 km spatial resolution: The use of training data derived from Landsat imagery in decision tree classifiers","volume":"19","author":"Hansen","year":"1998","journal-title":"Int. J. Remote Sens."},{"key":"ref_140","doi-asserted-by":"crossref","first-page":"850","DOI":"10.1126\/science.1244693","article-title":"High-resolution global maps of 21st-century forest cover change","volume":"342","author":"Hansen","year":"2013","journal-title":"Science"},{"key":"ref_141","doi-asserted-by":"crossref","first-page":"2897","DOI":"10.1016\/j.rse.2010.07.008","article-title":"Detecting trends in forest disturbance and recovery using yearly Landsat time series: 1. LandTrendr\u2014Temporal segmentation algorithms","volume":"114","author":"Kennedy","year":"2010","journal-title":"Remote Sens. Environ."},{"key":"ref_142","unstructured":"Oppenshaw, S., and Taylor, P. (1979). A million or so correlation coefficients. Statistical Methods in the Spatial Sciences, Pion."},{"key":"ref_143","doi-asserted-by":"crossref","first-page":"129","DOI":"10.1007\/BF02447512","article-title":"The modifiable areal unit problem and implications for landscape ecology","volume":"11","author":"Jelinski","year":"1996","journal-title":"Landsc. Ecol."},{"key":"ref_144","unstructured":"Weiss, M., de Beaufort, L., Baret, F., Allard, D., Bruguier, N., and Marloie, O. (2001, January 8\u201312). Mapping leaf area index measurements at different scales for the validation of large swath satellite sensors: First results of the VALERI project. Proceedings of the 8th International Symposium in Physical Measurements and Remote Sensing, Aussois, France."},{"key":"ref_145","doi-asserted-by":"crossref","first-page":"414","DOI":"10.1016\/S0034-4257(02)00047-0","article-title":"Multiscale analysis and validation of the MODIS LAI product: I. Uncertainty assessment","volume":"83","author":"Tian","year":"2002","journal-title":"Remote Sens. Environ."},{"key":"ref_146","doi-asserted-by":"crossref","unstructured":"Ramachandran, B., Justice, C.O., and Abrams, M.J. (2011). MODIS Land Data Products: Generation, Quality Assurance and Validation. Land Remote Sensing and Global Environmental Change: NASA\u2019s Earth Observing System and the Science of ASTER and MODIS, Springer New York.","DOI":"10.1007\/978-1-4419-6749-7"},{"key":"ref_147","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/S0034-4257(99)00053-X","article-title":"Validating MODIS terrestrial ecology products: linking in situ and satellite measurements","volume":"70","author":"Cohen","year":"1999","journal-title":"Remote Sens. Environ."},{"key":"ref_148","doi-asserted-by":"crossref","first-page":"1980","DOI":"10.1111\/gcb.12838","article-title":"Mapping global cropland and field size","volume":"21","author":"Fritz","year":"2015","journal-title":"Glob. Chang. Biol."},{"key":"ref_149","doi-asserted-by":"crossref","unstructured":"Debats, S.R., Estes, L.D., Thompson, D.R., and Caylor, K.K. (2017). Integrating Active Learning and Crowdsourcing into Large-Scale Supervised Landcover Mapping Algorithms. PeerJ, preprints.","DOI":"10.7287\/peerj.preprints.3004"},{"key":"ref_150","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1016\/j.envsoft.2016.01.011","article-title":"A Platform for Crowdsourcing the Creation of Representative, Accurate Landcover Maps","volume":"80","author":"Estes","year":"2016","journal-title":"Environ. Model. Softw."},{"key":"ref_151","doi-asserted-by":"crossref","first-page":"235","DOI":"10.1016\/j.rse.2018.10.039","article-title":"Conflation of expert and crowd reference data to validate global binary thematic maps","volume":"221","author":"Waldner","year":"2019","journal-title":"Remote Sens. Environ."},{"key":"ref_152","doi-asserted-by":"crossref","unstructured":"Bey, A., S\u00e1nchez-Paus D\u00edaz, A., Maniatis, D., Marchi, G., Mollicone, D., Ricci, S., Bastin, J.-F., Moore, R., Federici, S., and Rezende, M. (2016). Collect Earth: Land Use and Land Cover Assessment through Augmented Visual Interpretation. Remote Sens., 8.","DOI":"10.3390\/rs8100807"},{"key":"ref_153","unstructured":"Fritz, S., Sturn, T., Karner, M., Moorthy, I., See, L., Laso Bayas, J.C., and Fraisl, D. (2019, January 1\u20134). FotoQuest Go: A Citizen Science Approach to the Collection of In-Situ Land Cover and Land Use Data for Calibration and Validation. Proceedings of the Digital Earth Observation, Salzburg, Austria."},{"key":"ref_154","doi-asserted-by":"crossref","first-page":"2232","DOI":"10.1016\/j.rse.2011.04.022","article-title":"Using active learning to adapt remote sensing image classifiers","volume":"115","author":"Tuia","year":"2011","journal-title":"Remote Sens. Environ."},{"key":"ref_155","doi-asserted-by":"crossref","first-page":"754","DOI":"10.1080\/01431161.2013.873152","article-title":"Variability of operator performance in remote-sensing image interpretation: the importance of human and external factors","volume":"35","author":"Gardin","year":"2014","journal-title":"Int. J. Remote Sens."},{"key":"ref_156","doi-asserted-by":"crossref","first-page":"140","DOI":"10.1016\/j.apgeog.2015.12.006","article-title":"Integrating OpenStreetMap crowdsourced data and Landsat time-series imagery for rapid land use\/land cover (LULC) mapping: Case study of the Laguna de Bay area of the Philippines","volume":"67","author":"Johnson","year":"2016","journal-title":"Appl. Geogr."},{"key":"ref_157","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1016\/j.rse.2018.04.025","article-title":"Smallholder crop area mapped with wall-to-wall WorldView sub-meter panchromatic image texture: A test case for Tigray, Ethiopia","volume":"212","author":"Neigh","year":"2018","journal-title":"Remote Sens. Environ."},{"key":"ref_158","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1016\/j.rse.2012.08.013","article-title":"Land change for all municipalities in Latin America and the Caribbean assessed from 250-m MODIS imagery (2001\u20132010)","volume":"126","author":"Clark","year":"2012","journal-title":"Remote Sens. Environ."},{"key":"ref_159","doi-asserted-by":"crossref","first-page":"199","DOI":"10.1068\/b31135","article-title":"What is land cover?","volume":"32","author":"Comber","year":"2005","journal-title":"Environ. Plan."},{"key":"ref_160","doi-asserted-by":"crossref","first-page":"154","DOI":"10.1016\/j.compenvurbsys.2011.11.001","article-title":"An ontology of slums for image-based classification","volume":"36","author":"Kohli","year":"2012","journal-title":"Comput. Environ. Urban Syst."},{"key":"ref_161","doi-asserted-by":"crossref","first-page":"974","DOI":"10.1111\/j.1365-2486.2010.02307.x","article-title":"Challenges in using land use and land cover data for global change studies","volume":"17","author":"Verburg","year":"2011","journal-title":"Glob. Chang. Biol."},{"key":"ref_162","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1016\/j.rse.2011.02.030","article-title":"Remote sensing of impervious surfaces in the urban areas: Requirements, methods, and trends","volume":"117","author":"Weng","year":"2012","journal-title":"Remote Sens. Environ."},{"key":"ref_163","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1016\/j.compenvurbsys.2016.07.010","article-title":"Uncertainty analysis for image interpretations of urban slums","volume":"60","author":"Kohli","year":"2016","journal-title":"Comput. Environ. Urban Syst."},{"key":"ref_164","doi-asserted-by":"crossref","first-page":"125","DOI":"10.1016\/j.ecocom.2009.08.002","article-title":"While Boolean sets non-gently rip: A theoretical framework on fuzzy sets for mapping landscape patterns","volume":"7","author":"Rocchini","year":"2010","journal-title":"Ecol. Complex."},{"key":"ref_165","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1080\/136588100240895","article-title":"Fuzzy set theory and thematic maps: Accuracy assessment and area estimation","volume":"14","author":"Woodcock","year":"2000","journal-title":"Int. J. Geogr. Inf. Sci."},{"key":"ref_166","doi-asserted-by":"crossref","first-page":"128","DOI":"10.1016\/j.cageo.2012.05.022","article-title":"Uncertainty in ecosystem mapping by remote sensing","volume":"50","author":"Rocchini","year":"2013","journal-title":"Comput. Geosci."},{"key":"ref_167","doi-asserted-by":"crossref","first-page":"2721","DOI":"10.1080\/014311698214479","article-title":"A fuzzy classification of sub-urban land cover from remotely sensed imagery","volume":"19","author":"Zhang","year":"1998","journal-title":"Int. J. Remote Sens."},{"key":"ref_168","doi-asserted-by":"crossref","first-page":"311","DOI":"10.1016\/0034-4257(87)90015-0","article-title":"The factor of scale in remote sensing","volume":"21","author":"Woodcock","year":"1987","journal-title":"Remote Sens. Environ."},{"key":"ref_169","doi-asserted-by":"crossref","first-page":"2025","DOI":"10.1080\/014311698214848","article-title":"Review article Synergy in remote sensing-what\u2019s in a pixel?","volume":"19","author":"Cracknell","year":"1998","journal-title":"Int. J. Remote Sens."},{"key":"ref_170","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1080\/13658810500391024","article-title":"A generalized cross-tabulation matrix to compare soft-classified maps at multiple resolutions","volume":"20","author":"Pontius","year":"2006","journal-title":"Int. J. Geogr. Inf. Sci."},{"key":"ref_171","doi-asserted-by":"crossref","first-page":"1081","DOI":"10.1016\/j.rse.2007.07.017","article-title":"Sub-pixel confusion\u2013uncertainty matrix for assessing soft classifications","volume":"112","author":"Wang","year":"2008","journal-title":"Remote Sens. Environ."},{"key":"ref_172","first-page":"443","article-title":"The continuum of classification fuzziness in thematic mapping","volume":"65","author":"Foody","year":"1999","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_173","doi-asserted-by":"crossref","first-page":"238","DOI":"10.1007\/BF01424229","article-title":"Fully fuzzy supervised classification of land cover from remotely sensed imagery with an artificial neural network","volume":"5","author":"Foody","year":"1997","journal-title":"Neural Comput. Appl."},{"key":"ref_174","doi-asserted-by":"crossref","unstructured":"Laso Bayas, J.C., See, L., Fritz, S., Sturn, T., Perger, C., D\u00fcrauer, M., Karner, M., Moorthy, I., Schepaschenko, D., and Domian, D. (2016). Crowdsourcing In-Situ Data on Land Cover and Land Use Using Gamification and Mobile Technology. Remote Sens., 8.","DOI":"10.3390\/rs8110905"},{"key":"ref_175","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.rse.2015.01.006","article-title":"A critical synthesis of remotely sensed optical image change detection techniques","volume":"160","author":"Tewkesbury","year":"2015","journal-title":"Remote Sens. Environ."},{"key":"ref_176","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1016\/j.rse.2018.04.014","article-title":"Using volunteered geographic information (VGI) in design-based statistical inference for area estimation and accuracy assessment of land cover","volume":"212","author":"Stehman","year":"2018","journal-title":"Remote Sens. Environ."},{"key":"ref_177","doi-asserted-by":"crossref","first-page":"208","DOI":"10.1016\/j.foreco.2007.06.033","article-title":"Accuracy of forest inventory mapping: Some implications for boreal forest management","volume":"252","author":"Thompson","year":"2007","journal-title":"For. Ecol. Manag."},{"key":"ref_178","doi-asserted-by":"crossref","first-page":"1654","DOI":"10.1136\/bmj.312.7047.1654","article-title":"Statistics notes: Measurement error","volume":"312","author":"Bland","year":"1996","journal-title":"BMJ"},{"key":"ref_179","unstructured":"Martin, D. (2008). An Introduction to \u201cThe Guide to the Expression of Uncertainty in Measurement\u201d. Evaluation of Measurement Data\u2014Guide to the Expression of Uncertainty in Measurement, JCGM."},{"key":"ref_180","doi-asserted-by":"crossref","unstructured":"Thanh Noi, P., and Kappas, M. (2017). Comparison of Random Forest, k-Nearest Neighbor, and Support Vector Machine Classifiers for Land Cover Classification Using Sentinel-2 Imagery. Sensors, 18.","DOI":"10.3390\/s18010018"},{"key":"ref_181","unstructured":"Song, K. (2010). Tackling Uncertainties and Errors in the Satellite Monitoring of Forest Cover Change. [Ph.D. Thesis, University of Maryland]."},{"key":"ref_182","doi-asserted-by":"crossref","first-page":"3275","DOI":"10.1080\/01431160902755346","article-title":"The impact of imperfect ground reference data on the accuracy of land cover change estimation","volume":"30","author":"Foody","year":"2009","journal-title":"Int. J. Remote Sens."},{"key":"ref_183","doi-asserted-by":"crossref","first-page":"783","DOI":"10.1080\/2150704X.2013.798708","article-title":"Ground reference data error and the mis-estimation of the area of land cover change as a function of its abundance","volume":"4","author":"Foody","year":"2013","journal-title":"Remote Sens. Lett."},{"key":"ref_184","doi-asserted-by":"crossref","unstructured":"Homer, C.G., Fry, J.A., Barnes, C.A., and National land cover dataset (NLCD) (2012). The National Land Cover Database.","DOI":"10.3133\/fs20123020"},{"key":"ref_185","doi-asserted-by":"crossref","first-page":"014005","DOI":"10.1088\/1748-9326\/5\/1\/014005","article-title":"Radiative forcing and temperature response to changes in urban albedos and associated CO2 offsets","volume":"5","author":"Menon","year":"2010","journal-title":"Environ. Res. Lett."},{"key":"ref_186","doi-asserted-by":"crossref","first-page":"83","DOI":"10.1016\/j.landurbplan.2011.06.004","article-title":"Carbon consequences of land cover change and expansion of urban lands: A case study in the Seattle metropolitan region","volume":"103","author":"Hutyra","year":"2011","journal-title":"Landsc. Urban Plan."},{"key":"ref_187","doi-asserted-by":"crossref","first-page":"512","DOI":"10.1016\/j.scitotenv.2015.12.033","article-title":"Assessing the global warming potential of human settlement expansion in a mesic temperate landscape from 2005 to 2050","volume":"545-546","author":"Reinmann","year":"2016","journal-title":"Sci. Total Environ."},{"key":"ref_188","doi-asserted-by":"crossref","first-page":"366","DOI":"10.1016\/j.scitotenv.2017.03.028","article-title":"Accounting for urban biogenic fluxes in regional carbon budgets","volume":"592","author":"Hardiman","year":"2017","journal-title":"Sci. Total Environ."},{"key":"ref_189","doi-asserted-by":"crossref","first-page":"16083","DOI":"10.1073\/pnas.1211658109","article-title":"Global forecasts of urban expansion to 2030 and direct impacts on biodiversity and carbon pools","volume":"109","author":"Seto","year":"2012","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_190","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1016\/j.progress.2011.04.001","article-title":"The dimensions of global urban expansion: Estimates and projections for all countries, 2000\u20132050","volume":"75","author":"Angel","year":"2011","journal-title":"Prog. Plann."},{"key":"ref_191","doi-asserted-by":"crossref","first-page":"715","DOI":"10.14358\/PERS.78.7.715","article-title":"Modeling percent tree canopy cover: A pilot study","volume":"78","author":"Coulston","year":"2012","journal-title":"Photogramm. Eng Remote Sens."},{"key":"ref_192","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1073\/pnas.1612369114","article-title":"Edge effects enhance carbon uptake and its vulnerability to climate change in temperate broadleaf forests","volume":"114","author":"Reinmann","year":"2017","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_193","unstructured":"Rolnick, D., Veit, A., Belongie, S., and Shavit, N. (2017). Deep Learning is Robust to Massive Label Noise. arXiv."},{"key":"ref_194","unstructured":"Nachmany, Y., and Alemohammad, H. (2019, January 16\u201320). Detecting Roads from Satellite Imagery in the Developing World. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Long Beach, CA, USA."},{"key":"ref_195","unstructured":"The SpaceNet Catalog SpaceNet on Amazon Web Services (AWS) (2019, November 15). \u201cDatasets.\u201d The SpaceNet Catalog. Available online: https:\/\/spacenetchallenge.github.io\/datasets\/datasetHomePage.html."},{"key":"ref_196","doi-asserted-by":"crossref","first-page":"4101","DOI":"10.5194\/bg-14-4101-2017","article-title":"Water, Energy, and Carbon with Artificial Neural Networks (WECANN): A statistically-based estimate of global surface turbulent fluxes and gross primary productivity using solar-induced fluorescence","volume":"14","author":"Alemohammad","year":"2017","journal-title":"Biogeosciences"},{"key":"ref_197","doi-asserted-by":"crossref","first-page":"6229","DOI":"10.1002\/2014GL061322","article-title":"Extended triple collocation: Estimating errors and correlation coefficients with respect to an unknown target","volume":"41","author":"McColl","year":"2014","journal-title":"Geophys. Res. Lett."},{"key":"ref_198","doi-asserted-by":"crossref","first-page":"210","DOI":"10.1016\/j.rse.2016.03.010","article-title":"A Generalized Computer Vision Approach to Mapping Crop Fields in Heterogeneous Agricultural Landscapes","volume":"179","author":"Debats","year":"2016","journal-title":"Remote Sens. Environ."},{"key":"ref_199","unstructured":"Estes, L.D., Ye, S., Song, L., Avery, R.B., McRitchie, D., Eastman, J.R., and Debats, S.R. (2019). Improving Maps of Smallholder-Dominated Croplands through Tight Integration of Human and Machine Intelligence, American Geophysical Union."},{"key":"ref_200","doi-asserted-by":"crossref","first-page":"931","DOI":"10.1038\/s41893-019-0396-x","article-title":"The impact of agricultural interventions can be doubled by using satellite data","volume":"2","author":"Jain","year":"2019","journal-title":"Nat. Sustain."},{"key":"ref_201","doi-asserted-by":"crossref","unstructured":"Pontius, R.G. (2018). Criteria to Confirm Models that Simulate Deforestation and Carbon Disturbance. Land, 7.","DOI":"10.3390\/land7030105"},{"key":"ref_202","doi-asserted-by":"crossref","first-page":"341","DOI":"10.1146\/annurev-economics-080315-015058","article-title":"Recent Advances in the Measurement Error Literature","volume":"8","author":"Schennach","year":"2016","journal-title":"Annu. Rev. Econom."},{"key":"ref_203","doi-asserted-by":"crossref","first-page":"3196","DOI":"10.1080\/01431161.2016.1194545","article-title":"Towards a set of agrosystem-specific cropland mapping methods to address the global cropland diversity","volume":"37","author":"Waldner","year":"2016","journal-title":"Int. J. Remote Sens."},{"key":"ref_204","unstructured":"Castelluccio, M., Poggi, G., Sansone, C., and Verdoliva, L. (2015). Land Use Classification in Remote Sensing Images by Convolutional Neural Networks. arXiv."},{"key":"ref_205","unstructured":"Azevedo, T., Souza, C.M., Shimbo, J., and Alencar, A. (2018). MapBiomas Initiative: Mapping Annual Land Cover and Land Use Changes in Brazil from 1985 to 2017, American Geophysical Union."},{"key":"ref_206","doi-asserted-by":"crossref","unstructured":"Brown, J.F., Tollerud, H.J., Barber, C.P., Zhou, Q., Dwyer, J.L., Vogelmann, J.E., Loveland, T.R., Woodcock, C.E., Stehman, S.V., and Zhu, Z. (2019). Lessons learned implementing an operational continuous United States national land change monitoring capability: The Land Change Monitoring, Assessment, and Projection (LCMAP) approach. Remote Sens. Environ., 111356.","DOI":"10.1016\/j.rse.2019.111356"},{"key":"ref_207","doi-asserted-by":"crossref","first-page":"819","DOI":"10.1038\/s41559-018-0524-4","article-title":"The spatial and temporal domains of modern ecology","volume":"2","author":"Estes","year":"2018","journal-title":"Nat. Ecol. Evol."},{"key":"ref_208","first-page":"611","article-title":"Remote sensing of urban\/suburban infrastructure and socio-economic attributes","volume":"65","author":"Jensen","year":"1999","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_209","doi-asserted-by":"crossref","first-page":"1157","DOI":"10.3390\/rs3061157","article-title":"Strategies for Incorporating High-Resolution Google Earth Databases to Guide and Validate Classifications: Understanding Deforestation in Borneo","volume":"3","author":"Dorais","year":"2011","journal-title":"Remote Sens."},{"key":"ref_210","doi-asserted-by":"crossref","first-page":"246","DOI":"10.1016\/j.rse.2012.10.010","article-title":"Long-term land cover dynamics by multi-temporal classification across the Landsat-5 record","volume":"128","author":"Sexton","year":"2013","journal-title":"Remote Sens. Environ."},{"key":"ref_211","doi-asserted-by":"crossref","unstructured":"Reis, M.S., Escada, M.I.S., Dutra, L.V., Sant\u2019Anna, S.J.S., and Vogt, N.D. (2018). Towards a Reproducible LULC Hierarchical Class Legend for Use in the Southwest of Par\u00e1 State, Brazil: A Comparison with Remote Sensing Data-Driven Hierarchies. Land, 7.","DOI":"10.3390\/land7020065"},{"key":"ref_212","doi-asserted-by":"crossref","unstructured":"Anderson, J.R. (1976). A Land Use and Land Cover Classification System for Use with Remote Sensor Data.","DOI":"10.3133\/pp964"},{"key":"ref_213","doi-asserted-by":"crossref","first-page":"1719","DOI":"10.1109\/TGRS.2006.871219","article-title":"A joint initiative for harmonization and validation of land cover datasets","volume":"44","author":"Herold","year":"2006","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_214","doi-asserted-by":"crossref","first-page":"593","DOI":"10.1093\/jae\/ejv011","article-title":"From Guesstimates to GPStimates: Land Area Measurement and Implications for Agricultural Analysis","volume":"24","author":"Carletto","year":"2015","journal-title":"J. Afr. Econ."},{"key":"ref_215","doi-asserted-by":"crossref","unstructured":"See, L., Comber, A., Salk, C., Fritz, S., van der Velde, M., Perger, C., Schill, C., McCallum, I., Kraxner, F., and Obersteiner, M. (2013). Comparing the quality of crowdsourced data contributed by expert and non-experts. PLoS ONE, 8.","DOI":"10.1371\/journal.pone.0069958"},{"key":"ref_216","doi-asserted-by":"crossref","first-page":"3457","DOI":"10.1080\/014311698214136","article-title":"A framework for selecting appropriate remotely sensed data dimensions for environmental monitoring and management","volume":"19","author":"Phinn","year":"1998","journal-title":"Int. J. Remote Sens."},{"key":"ref_217","doi-asserted-by":"crossref","first-page":"117","DOI":"10.1016\/S0034-4257(00)00087-0","article-title":"Optimizing Remotely Sensed Solutions for Monitoring, Modeling, and Managing Coastal Environments","volume":"73","author":"Phinn","year":"2000","journal-title":"Remote Sens. Environ."},{"key":"ref_218","doi-asserted-by":"crossref","first-page":"823","DOI":"10.1080\/01431160600746456","article-title":"A survey of image classification methods and techniques for improving classification performance","volume":"28","author":"Lu","year":"2007","journal-title":"Int. J. Remote Sens."},{"key":"ref_219","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1016\/j.rse.2004.05.008","article-title":"Mapping vegetation in a heterogeneous mountain rangeland using landsat data: An alternative method to define and classify land-cover units","volume":"92","author":"Cingolani","year":"2004","journal-title":"Remote Sens. Environ."},{"key":"ref_220","doi-asserted-by":"crossref","unstructured":"Burke, M., and Lobell, D.B. (2017). Satellite-based assessment of yield variation and its determinants in smallholder African systems. Proc. Natl. Acad. Sci. USA.","DOI":"10.1073\/pnas.1616919114"},{"key":"ref_221","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1016\/j.rse.2019.04.016","article-title":"Smallholder maize area and yield mapping at national scales with Google Earth Engine","volume":"228","author":"Jin","year":"2019","journal-title":"Remote Sens. Environ."},{"key":"ref_222","doi-asserted-by":"crossref","first-page":"324","DOI":"10.1016\/j.rse.2015.04.021","article-title":"A Scalable Satellite-Based Crop Yield Mapper","volume":"164","author":"Lobell","year":"2015","journal-title":"Remote Sens. Environ."},{"key":"ref_223","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1016\/j.fcr.2015.03.004","article-title":"How Good Is Good Enough? Data Requirements for Reliable Crop Yield Simulations and Yield-Gap Analysis","volume":"177","author":"Grassini","year":"2015","journal-title":"Field Crops Res."},{"key":"ref_224","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","article-title":"ImageNet Large Scale Visual Recognition Challenge","volume":"115","author":"Russakovsky","year":"2015","journal-title":"Int. J. Comput. Vis."},{"key":"ref_225","doi-asserted-by":"crossref","first-page":"162","DOI":"10.1007\/978-3-319-16865-4_11","article-title":"NOKMeans: Non-Orthogonal K-means Hashing","volume":"Volume 9003","author":"Cremers","year":"2015","journal-title":"Computer Vision\u2014ACCV 2014"},{"key":"ref_226","doi-asserted-by":"crossref","unstructured":"Basu, S., Ganguly, S., Mukhopadhyay, S., DiBiano, R., Karki, M., and Nemani, R. (2015, January 3\u20136). DeepSat: A Learning Framework for Satellite Imagery. Proceedings of the Proceedings of the 23rd SIGSPATIAL International Conference on Advances in Geographic Information Systems, Seattle, WA, USA.","DOI":"10.1145\/2820783.2820816"},{"key":"ref_227","doi-asserted-by":"crossref","unstructured":"Yang, Y., and Newsam, S. (2010, January 2). Bag-of-visual-words and spatial extensions for land-use classification. Proceedings of the 18th SIGSPATIAL International Conference on Advances in Geographic Information Systems, San Jose, CA, USA.","DOI":"10.1145\/1869790.1869829"},{"key":"ref_228","doi-asserted-by":"crossref","first-page":"8558","DOI":"10.1029\/2018WR022643","article-title":"A Transdisciplinary Review of Deep Learning Research and Its Relevance for Water Resources Scientists","volume":"54","author":"Shen","year":"2018","journal-title":"Water Resour. Res."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/12\/6\/1034\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:10:53Z","timestamp":1760173853000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/12\/6\/1034"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,3,23]]},"references-count":228,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2020,3]]}},"alternative-id":["rs12061034"],"URL":"https:\/\/doi.org\/10.3390\/rs12061034","relation":{"has-preprint":[{"id-type":"doi","id":"10.31223\/osf.io\/7zsyr","asserted-by":"object"}]},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,3,23]]}}}