{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,17]],"date-time":"2026-03-17T18:31:18Z","timestamp":1773772278784,"version":"3.50.1"},"reference-count":79,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2020,4,7]],"date-time":"2020-04-07T00:00:00Z","timestamp":1586217600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100000780","name":"European Commission","doi-asserted-by":"publisher","award":["689812"],"award-info":[{"award-number":["689812"]}],"id":[{"id":"10.13039\/501100000780","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Land use and land cover (LULC) mapping is often undertaken by national mapping agencies, where these LULC products are used for different types of monitoring and reporting applications. Updating of LULC databases is often done on a multi-year cycle due to the high costs involved, so changes are only detected when mapping exercises are repeated. Consequently, the information on LULC can quickly become outdated and hence may be incorrect in some areas. In the current era of big data and Earth observation, change detection algorithms can be used to identify changes in urban areas, which can then be used to automatically update LULC databases on a more continuous basis. However, the change detection algorithm must be validated before the changes can be committed to authoritative databases such as those produced by national mapping agencies. This paper outlines a change detection algorithm for identifying construction sites, which represent ongoing changes in LU, developed in the framework of the LandSense project. We then use volunteered geographic information (VGI) captured through the use of mapathons from a range of different groups of contributors to validate these changes. In total, 105 contributors were involved in the mapathons, producing a total of 2778 observations. The 105 contributors were grouped according to six different user-profiles and were analyzed to understand the impact of the experience of the users on the accuracy assessment. Overall, the results show that the change detection algorithm is able to identify changes in residential land use to an adequate level of accuracy (85%) but changes in infrastructure and industrial sites had lower accuracies (57% and 75 %, respectively), requiring further improvements. In terms of user profiles, the experts in LULC from local authorities, researchers in LULC at the French national mapping agency (IGN), and first-year students with a basic knowledge of geographic information systems had the highest overall accuracies (86.2%, 93.2%, and 85.2%, respectively). Differences in how the users approach the task also emerged, e.g., local authorities used knowledge and context to try to identify types of change while those with no knowledge of LULC (i.e., normal citizens) were quicker to choose \u2018Unknown\u2019 when the visual interpretation of a class was more difficult.<\/jats:p>","DOI":"10.3390\/rs12071186","type":"journal-article","created":{"date-parts":[[2020,4,8]],"date-time":"2020-04-08T05:59:47Z","timestamp":1586325587000},"page":"1186","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":18,"title":["Use of Automated Change Detection and VGI Sources for Identifying and Validating Urban Land Use Change"],"prefix":"10.3390","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1101-1333","authenticated-orcid":false,"given":"A.-M.","family":"Olteanu-Raimond","sequence":"first","affiliation":[{"name":"LASTIG, Univ Gustave Eiffel, ENSG, IGN, F-94160 Saint-Mande, France"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2665-7065","authenticated-orcid":false,"given":"L.","family":"See","sequence":"additional","affiliation":[{"name":"International Institute for Applied Systems Analysis (IIASA), 2361 Laxenburg, Austria"}]},{"given":"M.","family":"Schultz","sequence":"additional","affiliation":[{"name":"GI-Science, Heidelberg University, 69120 Heidelberg, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6464-3054","authenticated-orcid":false,"given":"G.","family":"Foody","sequence":"additional","affiliation":[{"name":"School of Geography, University of Nottingham, Nottingham NG7 2RD, UK"}]},{"given":"M.","family":"Riffler","sequence":"additional","affiliation":[{"name":"GeoVille GmbH, 6020 Innsbruck, Austria"}]},{"given":"T.","family":"Gasber","sequence":"additional","affiliation":[{"name":"GeoVille GmbH, 6020 Innsbruck, Austria"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3090-5076","authenticated-orcid":false,"given":"L.","family":"Jolivet","sequence":"additional","affiliation":[{"name":"LASTIG, Univ Gustave Eiffel, ENSG, IGN, F-94160 Saint-Mande, France"}]},{"given":"A.","family":"le Bris","sequence":"additional","affiliation":[{"name":"LASTIG, Univ Gustave Eiffel, ENSG, IGN, F-94160 Saint-Mande, France"}]},{"given":"Y.","family":"Meneroux","sequence":"additional","affiliation":[{"name":"LASTIG, Univ Gustave Eiffel, ENSG, IGN, F-94160 Saint-Mande, France"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2001-7542","authenticated-orcid":false,"given":"L.","family":"Liu","sequence":"additional","affiliation":[{"name":"LASTIG, Univ Gustave Eiffel, ENSG, IGN, F-94160 Saint-Mande, France"}]},{"given":"M.","family":"Poup\u00e9e","sequence":"additional","affiliation":[{"name":"LASTIG, Univ Gustave Eiffel, ENSG, IGN, F-77420 Champs-sur-Marne, France"}]},{"given":"M.","family":"Gombert","sequence":"additional","affiliation":[{"name":"Direction of Operations and Territory, 33 000 Bordeaux, IGN, France"}]}],"member":"1968","published-online":{"date-parts":[[2020,4,7]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"2145","DOI":"10.5194\/bg-9-2145-2012","article-title":"Revisiting land cover observation to address the needs of the climate modeling community","volume":"9","author":"Bontemps","year":"2012","journal-title":"Biogeosciences"},{"key":"ref_2","unstructured":"EEA (2019). Land Take in Europe, European Environment Agency."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Manakos, I., and Braun, M. (2014). Land Use and Land Cover Mapping in Europe: Practices & Trends, Springer. Remote Sensing and Digital Image Processing.","DOI":"10.1007\/978-94-007-7969-3"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Theobald, D.M. (2014). Development and applications of a comprehensive land use classification and map for the US. PLoS ONE, 9.","DOI":"10.1371\/journal.pone.0094628"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"63","DOI":"10.1080\/17538947.2011.565080","article-title":"Land cover mapping applications with MODIS: A literature review","volume":"5","author":"Mas","year":"2012","journal-title":"Int. J. Digit. Earth"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Herold, M., See, L., Tsendbazar, N.-E., and Fritz, S. (2016). Towards an integrated global land cover monitoring and mapping system. Remote Sens., 8.","DOI":"10.3390\/rs8121036"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"2","DOI":"10.1016\/j.rse.2012.01.010","article-title":"Opening the archive: How free data has enabled the science and monitoring promise of Landsat","volume":"122","author":"Wulder","year":"2012","journal-title":"Remote Sens. Environ."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"160","DOI":"10.1038\/508160a","article-title":"Earth observation enters next phase","volume":"508","author":"Butler","year":"2014","journal-title":"Nature"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Manakos, I., and Braun, M. (2014). CORINE land cover and land cover change products. Land Use and Land Cover Mapping in Europe: Practices & Trends, Springer. Remote Sensing and Digital Image Processing.","DOI":"10.1007\/978-94-007-7969-3"},{"key":"ref_10","unstructured":"EEA (2014, April 15). Mapping Guide for a European Urban Atlas. Available online: http:\/\/www.eea.europa.eu\/data-and-maps\/data\/urban-atlas."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Foody, G.M., See, L., Fritz, S., Fonte, C.C., Mooney, P., Olteanu-Raimond, A.-M., and Antoniou, V. (2017). A review of OpenStreetMap data. Mapping and the Citizen Sensor, Ubiquity Press.","DOI":"10.5334\/bbf.c"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"See, L., Mooney, P., Foody, G., Bastin, L., Comber, A., Estima, J., Fritz, S., Kerle, N., Jiang, B., and Laakso, M. (2016). Crowdsourcing, citizen science or Volunteered Geographic Information? The current state of crowdsourced geographic information. ISPRS Int. J. Geo-Inf., 5.","DOI":"10.3390\/ijgi5050055"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/s10708-007-9111-y","article-title":"Citizens as sensors: The world of volunteered geography","volume":"69","author":"Goodchild","year":"2007","journal-title":"GeoJournal"},{"key":"ref_14","first-page":"1","article-title":"The rise of crowdsourcing","volume":"14","author":"Howe","year":"2006","journal-title":"Wired Mag."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"977","DOI":"10.1525\/bio.2009.59.11.9","article-title":"Citizen science: A developing tool for expanding science knowledge and scientific literacy","volume":"59","author":"Bonney","year":"2009","journal-title":"BioScience"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"682","DOI":"10.1068\/b35097","article-title":"How good is volunteered geographical information? A comparative study of OpenStreetMap and Ordnance Survey datasets","volume":"37","author":"Haklay","year":"2010","journal-title":"Environ. Plan. B Plan. Des."},{"key":"ref_17","unstructured":"Zielstra, D., and Zipf, A. (2010, January 11\u201314). A comparative study of proprietary geodata and volunteered geographic information for Germany. Proceedings of the 13th AGILE International Conference on Geographic Information Science, Guimar\u00e3es, Portugal."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"435","DOI":"10.1111\/j.1467-9671.2010.01203.x","article-title":"Quality assessment of the French OpenStreetMap dataset","volume":"14","author":"Girres","year":"2010","journal-title":"Trans. GIS"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Fonte, C., Minghini, M., Patriarca, J., Antoniou, V., See, L., and Skopeliti, A. (2017). Generating up-to-date and detailed land use and land cover maps ssing OpenStreetMap and GlobeLand30. ISPRS Int. J. Geo-Inf., 6.","DOI":"10.3390\/ijgi6040125"},{"key":"ref_20","first-page":"206","article-title":"Open land cover from OpenStreetMap and remote sensing","volume":"63","author":"Schultz","year":"2017","journal-title":"Int. J. Appl. Earth Obs. Geoinf."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"294","DOI":"10.1109\/TIP.2004.838698","article-title":"Image change detection algorithms: A systematic survey","volume":"14","author":"Radke","year":"2005","journal-title":"IEEE Trans. Image Process."},{"key":"ref_22","first-page":"145","article-title":"Building change detection from digital surface models and multi-spectral images","volume":"36","author":"Rottensteiner","year":"2007","journal-title":"Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Gressin, A., Vincent, N., Mallet, C., and Paparoditis, N. (2014, January 27\u201330). A unified framework for land-cover database update and enrichment using satellite imagery. Proceedings of the IEEE International Conference on Image Processing (ICIP), Paris, France.","DOI":"10.1109\/ICIP.2014.7026024"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"989","DOI":"10.1080\/01431168908903939","article-title":"Review Article Digital change detection techniques using remotely-sensed data","volume":"10","author":"Singh","year":"1989","journal-title":"Int. J. Remote Sens."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"2365","DOI":"10.1080\/0143116031000139863","article-title":"Change detection techniques","volume":"25","author":"Lu","year":"2004","journal-title":"Int. J. Remote Sens."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"850","DOI":"10.1126\/science.1244693","article-title":"High-resolution global maps of 21st-century forest cover change","volume":"342","author":"Hansen","year":"2013","journal-title":"Science"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Chen, J., Chen, J., Liu, H., and Peng, S. (2018). Detection of cropland change using multi-harmonic based phenological trajectory similarity. Remote Sens., 10.","DOI":"10.3390\/rs10071020"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"6635","DOI":"10.1080\/01431161.2019.1583394","article-title":"Change detection in urban areas from remote sensing data: A multidimensional classification scheme","volume":"40","author":"Rezgui","year":"2019","journal-title":"Int. J. Remote Sens."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Wang, Z., Yao, W., Tang, Q., Liu, L., Xiao, P., Kong, X., Zhang, P., Shi, F., and Wang, Y. (2018). Continuous change detection of forest\/grassland and cropland in the Loess Plateau of China using all available Landsat data. Remote Sens., 10.","DOI":"10.3390\/rs10111775"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"42","DOI":"10.1016\/j.rse.2014.02.015","article-title":"Good practices for estimating area and assessing accuracy of land change","volume":"148","author":"Olofsson","year":"2014","journal-title":"Remote Sens. Environ."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1305","DOI":"10.1109\/JSTARS.2013.2250257","article-title":"Using volunteered data in land cover map validation: Mapping West African forests","volume":"6","author":"Foody","year":"2013","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1269","DOI":"10.1080\/13658816.2015.1018266","article-title":"Usability of VGI for validation of land cover maps","volume":"29","author":"Fonte","year":"2015","journal-title":"Int. J. Geogr. Inf. Sci."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"See, L., Laso Bayas, J.-C., Schepaschenko, D., Perger, C., Dresel, C., Maus, V., Salk, C., Weichselbaum, J., Lesiv, M., and McCallum, I. (2017). LACO-Wiki: A new online land cover validation tool demonstrated using GlobeLand30 for Kenya. Remote Sens., 9.","DOI":"10.3390\/rs9070754"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Buchroithner, M., Prechtel, N., and Burghardt, D. (2014). G\u00e9oPeuple: The creation and the analysis of topographic and demographic data over 200 Years. Cartography from Pole to Pole, Springer Berlin Heidelberg.","DOI":"10.1007\/978-3-642-32618-9"},{"key":"ref_35","unstructured":"Moorthy, I., Fritz, S., See, L., and McCallum, I. (2017, January 23\u201328). LandSense: A citizen observatory and innovation marketplace for land use and land cover monitoring. Proceedings of the 19th EGU General Assembly, Vienna, Austria."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"160","DOI":"10.1080\/19322909.2018.1463585","article-title":"Humanitarian mapping as library outreach: A case for community-oriented mapathons","volume":"12","author":"Quill","year":"2018","journal-title":"J. Web Librariansh."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"635","DOI":"10.1126\/science.aam6527","article-title":"The extent of forest in dryland biomes","volume":"356","author":"Bastin","year":"2017","journal-title":"Science"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"208","DOI":"10.1016\/j.rse.2015.02.011","article-title":"Development of a global hybrid forest mask through the synergy of remote sensing, crowdsourcing and FAO statistics","volume":"162","author":"Schepaschenko","year":"2015","journal-title":"Remote Sens. Environ."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1980","DOI":"10.1111\/gcb.12838","article-title":"Mapping global cropland and field size","volume":"21","author":"Fritz","year":"2015","journal-title":"Glob. Chang. Biol."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"639","DOI":"10.1038\/s41586-018-0411-9","article-title":"Global land change from 1982 to 2016","volume":"560","author":"Song","year":"2018","journal-title":"Nature"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"152","DOI":"10.1016\/j.rse.2014.01.011","article-title":"Continuous change detection and classification of land cover using all available Landsat data","volume":"144","author":"Zhu","year":"2014","journal-title":"Remote Sens. Environ."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"418","DOI":"10.1038\/nature20584","article-title":"High-resolution mapping of global surface water and its long-term changes","volume":"540","author":"Pekel","year":"2016","journal-title":"Nature"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"4254","DOI":"10.1080\/01431161.2018.1452075","article-title":"Land cover 2.0","volume":"39","author":"Wulder","year":"2018","journal-title":"Int. J. Remote Sens."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"111254","DOI":"10.1016\/j.rse.2019.111254","article-title":"Landsat-8 and Sentinel-2 burned area mapping\u2014A combined sensor multi-temporal change detection approach","volume":"231","author":"Roy","year":"2019","journal-title":"Remote Sens. Environ."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Poulain, V., Inglada, J., Spigai, M., Tourneret, J.-Y., and Marthon, P. (2009, January 12\u201317). Fusion of high resolution optical and SAR images with vector data bases for change detection. Proceedings of the IEEE International Geoscience and Remote Sensing Symposium, Cape Town, South Africa.","DOI":"10.1109\/IGARSS.2009.5417537"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1016\/j.inffus.2012.05.003","article-title":"Information fusion techniques for change detection from multi-temporal remote sensing images","volume":"14","author":"Du","year":"2013","journal-title":"Inf. Fusion"},{"key":"ref_47","first-page":"25","article-title":"Change detection in a topographic building database using submetric satellite images","volume":"38","author":"Chehata","year":"2011","journal-title":"Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci."},{"key":"ref_48","unstructured":"Gladstone, C.S., Gardiner, A., and Holland, D. (2012, January 7\u20139). A semi-automatic method for detecting changes to ordnance survey\u2014Topographic data in rural environments. Proceedings of the 4th GEOBIA, Rio de Janeiro, Brazil."},{"key":"ref_49","unstructured":"Domenech, E., and Mallet, C. (2014). Change Detection in High-Resolution Land Use\/Land Cover Geodatabases (at Object Level); EuroSDR Official Publication, Buchdruckerei Ernst Becvar."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"529","DOI":"10.5194\/isprs-archives-XLI-B7-529-2016","article-title":"Updating national topographic database using change detection methods","volume":"41","author":"Keinan","year":"2016","journal-title":"Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"1217","DOI":"10.3390\/rs2051217","article-title":"Automatic detection of buildings and changes in buildings for updating of maps","volume":"2","author":"Matikainen","year":"2010","journal-title":"Remote Sens."},{"key":"ref_52","first-page":"463","article-title":"The regularized iteratively reweighted MAD method for change detection in multi-hyperspectral data","volume":"16","author":"Nielsen","year":"2007","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_53","unstructured":"Gueguen, L., and Datcu, M. (2009, January 28\u201330). Mixed information measure. Application to change detection in Earth observation. Proceedings of the MultiTemp 2009, the 5th International Workshop on the Analysis of Multi-Temporal Remote Sensing Images, Mansfield, CT, USA."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"4503","DOI":"10.1109\/TGRS.2011.2141999","article-title":"Change detection based on information measure","volume":"49","author":"Gueguen","year":"2011","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Molina, I., Martinez, E., Morillo, C., Velasco, J., and Jara, A. (2016). Assessment of data fusion algorithms for Earth observation change detection processes. Sensors, 16.","DOI":"10.3390\/s16101621"},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"390","DOI":"10.1016\/j.rse.2004.04.001","article-title":"Automatic change detection by evidential fusion of change indices","volume":"91","author":"Seltz","year":"2004","journal-title":"Remote Sens. Environ."},{"key":"ref_57","doi-asserted-by":"crossref","unstructured":"Varghese, A., Gubbi, J., Ramaswamy, A., and Balamuralidhar, P. (2018, January 8\u201314). ChangeNet: A deep learning architecture for visual change detection. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-11012-3_10"},{"key":"ref_58","doi-asserted-by":"crossref","unstructured":"Pomente, A., Picchiani, M., and Del Frate, F. (2018, January 22\u201327). Sentinel-2 Change detection based on deep features. Proceedings of the IGARSS 2018 IEEE International Geoscience and Remote Sensing Symposium, Valencia, Spain.","DOI":"10.1109\/IGARSS.2018.8519195"},{"key":"ref_59","doi-asserted-by":"crossref","unstructured":"Caye Daudt, R., Le Saux, B., Boulch, A., and Gousseau, Y. (2018, January 22\u201327). Urban change detection for multispectral Earth observation using Convolutional Neural Networks. Proceedings of the IEEE International Geoscience and Remote Sensing Symposium (IGARSS\u20192018), Valencia, Spain.","DOI":"10.1109\/IGARSS.2018.8518015"},{"key":"ref_60","doi-asserted-by":"crossref","unstructured":"Papadomanolaki, M., Verma, S., Vakalopoulou, M., Gupta, S., and Karantzalos, K. (August, January 28). Detecting urban changes with recurrent neural networks from multitemporal Sentinel-2 data. Proceedings of the IEEE International Geoscience and Remote Sensing Symposium (IGARSS\u20192019), Yokohama, Japan.","DOI":"10.1109\/IGARSS.2019.8900330"},{"key":"ref_61","doi-asserted-by":"crossref","unstructured":"Lebedev, M.A., Vizilter, Y.V., Vygolov, O.V., Knyaz, V.A., and Rubis, A.Y. (2018). Change detection in remote sensing images using Conditional Adversarial Networks. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci.","DOI":"10.5194\/isprs-archives-XLII-2-565-2018"},{"key":"ref_62","doi-asserted-by":"crossref","unstructured":"Peng, D., Zhang, M., and Wanbing, G. (2019). End-to-End change detection for high resolution satellite images using improved UNet++. Remote Sens., 11.","DOI":"10.3390\/rs11111382"},{"key":"ref_63","doi-asserted-by":"crossref","unstructured":"Caye Daudt, R., Le Saux, B., Boulch, A., and Gousseau, Y. (2019). Multitask learning for large-scale semantic change detection. Comput. Vis. Image Underst., 187.","DOI":"10.1016\/j.cviu.2019.07.003"},{"key":"ref_64","doi-asserted-by":"crossref","first-page":"122","DOI":"10.1016\/j.rse.2012.10.031","article-title":"Making better use of accuracy data in land change studies: Estimating accuracy and area and quantifying uncertainty using stratified estimation","volume":"129","author":"Olofsson","year":"2013","journal-title":"Remote Sens. Environ."},{"key":"ref_65","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1016\/j.rse.2018.04.014","article-title":"Using volunteered geographic information (VGI) in design-based statistical inference for area estimation and accuracy assessment of land cover","volume":"212","author":"Stehman","year":"2018","journal-title":"Remote Sens. Environ."},{"key":"ref_66","doi-asserted-by":"crossref","first-page":"1588","DOI":"10.1080\/13658816.2019.1593422","article-title":"Crowdsourced geospatial data quality: Challenges and future directions","volume":"33","author":"Basiri","year":"2019","journal-title":"Int. J. Geogr. Inf. Sci."},{"key":"ref_67","doi-asserted-by":"crossref","unstructured":"Herfort, B., Li, H., Fendrich, S., Lautenbach, S., and Zipf, A. (2019). Mapping human settlements with higher accuracy and less volunteer efforts by combining crowdsourcing and deep learning. Remote Sens., 11.","DOI":"10.3390\/rs11151799"},{"key":"ref_68","doi-asserted-by":"crossref","unstructured":"Albuquerque, J., Herfort, B., and Eckle, M. (2016). The tasks of the crowd: A typology of tasks in geographic information crowdsourcing and a case study in humanitarian mapping. Remote Sens., 8.","DOI":"10.3390\/rs8100859"},{"key":"ref_69","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1016\/0034-4257(91)90048-B","article-title":"A review of assessing the accuracy of classifications of remotely sensed data","volume":"37","author":"Congalton","year":"1991","journal-title":"Remote Sens. Environ."},{"key":"ref_70","unstructured":"Strahler, A.H., Boschetti, L., Foody, G.M., Friedl, M.A., Hansen, M.C., Herold, M., Mayaux, P., Morisette, J., Stehman, S.V., and Woodcock, C.E. (2006). Global Land Cover Validation: Recommendations for Evaluation and Accuracy Assessment of Global Land Cover Maps, European Commission."},{"key":"ref_71","doi-asserted-by":"crossref","unstructured":"Doxani, G., Vermote, E., Roger, J.-C., Gascon, F., Adriaensen, S., Frantz, D., Hagolle, O., Hollstein, A., Kirches, G., and Li, F. (2018). Atmospheric correction inter-comparison exercise. Remote Sens., 10.","DOI":"10.3390\/rs10020352"},{"key":"ref_72","doi-asserted-by":"crossref","unstructured":"Gascon, F., Bouzinac, C., Th\u00e9paut, O., Jung, M., Francesconi, B., Louis, J., Lonjou, V., Lafrance, B., Massera, S., and Gaudel-Vacaresse, A. (2017). Copernicus Sentinel-2A calibration and products validation status. Remote Sens., 9.","DOI":"10.3390\/rs9060584"},{"key":"ref_73","unstructured":"ESA (2020, March 02). Sentinel Online: Data Product Quality Reports. Available online: https:\/\/sentinel.esa.int\/web\/sentinel\/data-product-quality-reports."},{"key":"ref_74","unstructured":"Leutner, B., Horning, N., Schwalb-Willmann, J., and Hijmans, R.J. (2020, January 10). Package \u2018RStoolbox\u2019. Available online: https:\/\/CRAN.R-project.org\/package=RStoolbox."},{"key":"ref_75","doi-asserted-by":"crossref","first-page":"411","DOI":"10.1080\/014311698216062","article-title":"Change vector analysis: A technique for the multispectral monitoring of land cover and condition","volume":"19","author":"Johnson","year":"1998","journal-title":"Int. J. Remote Sens."},{"key":"ref_76","doi-asserted-by":"crossref","first-page":"111199","DOI":"10.1016\/j.rse.2019.05.018","article-title":"Key issues in rigorous accuracy assessment of land cover products","volume":"231","author":"Stehman","year":"2019","journal-title":"Remote Sens. Environ."},{"key":"ref_77","doi-asserted-by":"crossref","first-page":"378","DOI":"10.1037\/h0031619","article-title":"Measuring nominal scale agreement among many raters","volume":"76","author":"Fleiss","year":"1971","journal-title":"Psychol. Bull."},{"key":"ref_78","first-page":"431","article-title":"Using known map category marginal frequencies to improve estimates of thematic map accuracy","volume":"48","author":"Card","year":"1982","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_79","doi-asserted-by":"crossref","unstructured":"Olteanu-Raimond, A.-M., Jolivet, L., Van Damme, M.-D., Royer, T., Fraval, L., See, L., Sturn, T., Karner, M., Moorthy, I., and Fritz, S. (2018). An experimental framework for integrating citizen and community science into land cover, land use, and land change detection processes in a National Mapping Agency. Land, 7.","DOI":"10.3390\/land7030103"}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/12\/7\/1186\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:16:20Z","timestamp":1760174180000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/12\/7\/1186"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,4,7]]},"references-count":79,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2020,4]]}},"alternative-id":["rs12071186"],"URL":"https:\/\/doi.org\/10.3390\/rs12071186","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,4,7]]}}}