{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,10]],"date-time":"2026-03-10T08:00:14Z","timestamp":1773129614750,"version":"3.50.1"},"reference-count":43,"publisher":"MDPI AG","issue":"18","license":[{"start":{"date-parts":[[2023,9,18]],"date-time":"2023-09-18T00:00:00Z","timestamp":1694995200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Natural Science Foundation of China","award":["42221002"],"award-info":[{"award-number":["42221002"]}]},{"name":"National Natural Science Foundation of China","award":["41631178"],"award-info":[{"award-number":["41631178"]}]},{"name":"National Natural Science Foundation of China","award":["23XD1404100"],"award-info":[{"award-number":["23XD1404100"]}]},{"name":"Shanghai Academic Research Leader Program","award":["42221002"],"award-info":[{"award-number":["42221002"]}]},{"name":"Shanghai Academic Research Leader Program","award":["41631178"],"award-info":[{"award-number":["41631178"]}]},{"name":"Shanghai Academic Research Leader Program","award":["23XD1404100"],"award-info":[{"award-number":["23XD1404100"]}]},{"name":"Fundamental Research Funds for the Central Universities of China","award":["42221002"],"award-info":[{"award-number":["42221002"]}]},{"name":"Fundamental Research Funds for the Central Universities of China","award":["41631178"],"award-info":[{"award-number":["41631178"]}]},{"name":"Fundamental Research Funds for the Central Universities of China","award":["23XD1404100"],"award-info":[{"award-number":["23XD1404100"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>The new type of multi-temporal global land use data with multiple classes is able to provide information on both the different land covers and their temporal changes; furthermore, it is able to contribute to many applications, such as those involving global climate and Earth ecosystem analyses. However, the current accuracy assessment methods have two limitations regarding multi-temporal land cover data that have multiple classes. First, multi-temporal land cover uses data from multiple phases, which is time-consuming and inefficient if evaluated one by one. Secondly, the conversion between different land cover classes increases the complexity of the sample stratification, and the assessments with different types of land cover suffer from inefficient sample stratification. In this paper, we propose a spatiotemporal stratified sampling method for stratifying the multi-temporal GlobeLand30 products for China. The changed and unchanged types of each class of data in the three periods are used to obtain a reasonable stratification. Then, the strata labels are simplified by using binary coding, i.e., a 1 or 0 representing a specified class or a nonspecified class, to improve the efficiency of the stratification. Additionally, the stratified sample size is determined by the combination of proportional allocation and empirical evaluation. The experimental results show that spatiotemporal stratified sampling is beneficial for increasing the sample size of the \u201cchange\u201d strata for multi-temporal data and can evaluate not only the accuracy and area of the data in a single data but also the accuracy and area of the data in a multi-period change type and an unchanged type. This work also provides a good reference for the assessment of multi-temporal data with multiple classes.<\/jats:p>","DOI":"10.3390\/rs15184593","type":"journal-article","created":{"date-parts":[[2023,9,19]],"date-time":"2023-09-19T02:47:52Z","timestamp":1695091672000},"page":"4593","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Assessing the Accuracy of Multi-Temporal GlobeLand30 Products in China Using a Spatiotemporal Stratified Sampling Method"],"prefix":"10.3390","volume":"15","author":[{"given":"Yali","family":"Gong","sequence":"first","affiliation":[{"name":"College of Surveying and Geo-Informatics, Tongji University, Shanghai 200092, China"},{"name":"Shanghai Key Laboratory of Space Mapping and Remote Sensing for Planetary Exploration, Tongji University, Shanghai 200092, China"}]},{"given":"Huan","family":"Xie","sequence":"additional","affiliation":[{"name":"College of Surveying and Geo-Informatics, Tongji University, Shanghai 200092, China"},{"name":"Shanghai Key Laboratory of Space Mapping and Remote Sensing for Planetary Exploration, Tongji University, Shanghai 200092, China"},{"name":"Shanghai Institute of Intelligent Science and Technology, Tongji University, Shanghai 200092, China"}]},{"given":"Shicheng","family":"Liao","sequence":"additional","affiliation":[{"name":"College of Surveying and Geo-Informatics, Tongji University, Shanghai 200092, China"},{"name":"Shanghai Key Laboratory of Space Mapping and Remote Sensing for Planetary Exploration, Tongji University, Shanghai 200092, China"}]},{"given":"Yao","family":"Lu","sequence":"additional","affiliation":[{"name":"College of Surveying and Geo-Informatics, Tongji University, Shanghai 200092, China"},{"name":"Shanghai Key Laboratory of Space Mapping and Remote Sensing for Planetary Exploration, Tongji University, Shanghai 200092, China"}]},{"given":"Yanmin","family":"Jin","sequence":"additional","affiliation":[{"name":"College of Surveying and Geo-Informatics, Tongji University, Shanghai 200092, China"},{"name":"Shanghai Key Laboratory of Space Mapping and Remote Sensing for Planetary Exploration, Tongji University, Shanghai 200092, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9759-8122","authenticated-orcid":false,"given":"Chao","family":"Wei","sequence":"additional","affiliation":[{"name":"College of Surveying and Geo-Informatics, Tongji University, Shanghai 200092, China"},{"name":"Shanghai Key Laboratory of Space Mapping and Remote Sensing for Planetary Exploration, Tongji University, Shanghai 200092, China"}]},{"given":"Xiaohua","family":"Tong","sequence":"additional","affiliation":[{"name":"College of Surveying and Geo-Informatics, Tongji University, Shanghai 200092, China"},{"name":"Shanghai Key Laboratory of Space Mapping and Remote Sensing for Planetary Exploration, Tongji University, Shanghai 200092, China"}]}],"member":"1968","published-online":{"date-parts":[[2023,9,18]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1541","DOI":"10.1175\/BAMS-D-11-00254.1","article-title":"The ESA climate change initiative: Satellite data records for essential climate variables","volume":"94","author":"Hollmann","year":"2013","journal-title":"Bull. Am. Meteorol. Soc."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Stehman, S.V., and Foody, G.M. (2019). Key issues in rigorous accuracy assessment of land cover products. Remote Sens. Environ., 231.","DOI":"10.1016\/j.rse.2019.05.018"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"5125","DOI":"10.5194\/bg-9-5125-2012","article-title":"Carbon emissions from land use and land-cover change","volume":"9","author":"Houghton","year":"2012","journal-title":"Biogeosciences"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"202","DOI":"10.1016\/j.rse.2019.02.003","article-title":"Near real-time monitoring of tropical forest disturbance: New algorithms and assessment framework","volume":"224","author":"Tang","year":"2019","journal-title":"Remote Sens. Environ."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1016\/j.rse.2019.02.015","article-title":"Current status of Landsat program, science, and applications","volume":"225","author":"Wulder","year":"2019","journal-title":"Remote Sens. Environ."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Potapov, P., Hansen, M.C., Pickens, A., Hernandez-Serna, A., Tyukavina, A., Turubanova, S., Zalles, V., Li, X., Khan, A., and Stolle, F. (2022). The Global 2000\u20132020 Land Cover and Land Use Change Dataset Derived from the Landsat Archive: First Results. Front. Remote Sens., 3.","DOI":"10.3389\/frsen.2022.856903"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"494","DOI":"10.1016\/j.cosust.2013.07.003","article-title":"Land system change and food security: Towards multi-scale land system solutions","volume":"5","author":"Verburg","year":"2013","journal-title":"Curr. Opin. Environ. Sustain."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"195","DOI":"10.1007\/s11442-014-1082-6","article-title":"Spatiotemporal characteristics, patterns, and causes of land-use changes in China since the late 1980s","volume":"24","author":"Liu","year":"2014","journal-title":"J. Geogr. Sci."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"989","DOI":"10.1038\/s41467-017-01038-w","article-title":"The impact of anthropogenic land use and land cover change on regional climate extremes","volume":"8","author":"Findell","year":"2017","journal-title":"Nat. Commun."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"579","DOI":"10.1007\/s11442-017-1394-4","article-title":"Assessment of multifunctional landscapes dynamics in the mountainous basin of the Mo River (Togo, West Africa)","volume":"27","author":"Diwediga","year":"2017","journal-title":"J. Geogr. Sci."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Yin, C., Zhao, W., and Pereira, P. (2022). Soil conservation service underpins sustainable development goals. Glob. Ecol. Conserv., 33.","DOI":"10.1016\/j.gecco.2021.e01974"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"4923","DOI":"10.1080\/01431161.2014.930207","article-title":"Estimating area and map accuracy for stratified random sampling when the strata are different from the map classes","volume":"35","author":"Stehman","year":"2014","journal-title":"Int. J. Remote Sens."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Congalton, R.G., and Green, K. (1999). Assessing the Accuracy of Remotely Sensed Data: Principles and Practices, Lewis Publishers.","DOI":"10.1201\/9781420048568"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"301","DOI":"10.1023\/A:1025138423071","article-title":"Introduction to special issue on map accuracy","volume":"10","author":"Stehman","year":"2003","journal-title":"Environ. Ecol. Stat."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"3019","DOI":"10.1080\/01431160310001619607","article-title":"Remote sensing and land cover area estimation","volume":"25","author":"Gallego","year":"2004","journal-title":"Int. J. Remote Sens."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"42","DOI":"10.1016\/j.rse.2014.02.015","article-title":"Good practices for estimating area and assessing accuracy of land change","volume":"148","author":"Olofsson","year":"2014","journal-title":"Remote Sens. Environ."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Wickham, J., Stehman, S.V., Sorenson, D.G., Gass, L., and Dewitz, J.A. (2023). Thematic accuracy assessment of the NLCD 2019 land cover for the conterminous United States. GISci. Remote Sens., 60.","DOI":"10.1080\/15481603.2023.2181143"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Wickham, J., Stehman, S.V., Sorenson, D.G., Gass, L., and Dewitz, J.A. (2021). Thematic accuracy assessment of the NLCD 2016 land cover for the conterminous United States. Remote Sens. Environ., 257.","DOI":"10.1016\/j.rse.2021.112357"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Ar\u00e9valo, P., Olofsson, P., and Woodcock, C.E. (2020). Continuous monitoring of land change activities and post-disturbance dynamics from Landsat time series: A test methodology for REDD+ reporting. Remote Sens. Environ., 238.","DOI":"10.1016\/j.rse.2019.01.013"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1016\/j.isprsjprs.2014.09.002","article-title":"Global land cover mapping at 30 m resolution: A POK-based operational approach","volume":"103","author":"Chen","year":"2015","journal-title":"Isprs J. Photogramm. Remote Sens."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Chughtai, A.H., Abbasi, H., and Karas, I.R. (2021). A review on change detection method and accuracy assessment for land use land cover. Remote Sens. Appl. Soc. Environ., 22.","DOI":"10.1016\/j.rsase.2021.100482"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1570","DOI":"10.1016\/j.cageo.2011.02.006","article-title":"Designing a two-rank acceptance sampling plan for quality inspection of geospatial data products","volume":"37","author":"Tong","year":"2011","journal-title":"Comput. Geosci."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Olofsson, P., Ar\u00e9valo, P., Espejo, A.B., Green, C., Lindquist, E., McRoberts, R.E., and Sanz, M.J. (2020). Mitigating the effects of omission errors on area and area change estimates. Remote Sens. Environ., 236.","DOI":"10.1016\/j.rse.2019.111492"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Hao, X., Qiu, Y., Jia, G., Menenti, M., Ma, J., and Jiang, Z. (2023). Evaluation of Global Land Use\u2014Land Cover Data Products in Guangxi, China. Remote Sens., 15.","DOI":"10.3390\/rs15051291"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Auch, R.F., Wellington, D.F., Taylor, J.L., Stehman, S.V., Tollerud, H.J., Brown, J.F., Loveland, T.R., Pengra, B.W., Horton, J.A., and Zhu, Z. (2022). Conterminous United States Land-Cover Change (1985\u20132016): New Insights from Annual Time Series. Land, 11.","DOI":"10.3390\/land11020298"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1038\/s43016-021-00429-z","article-title":"Global maps of cropland extent and change show accelerated cropland expansion in the twenty-first century","volume":"3","author":"Potapov","year":"2022","journal-title":"Nat. Food"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"294","DOI":"10.1016\/j.rse.2012.12.001","article-title":"Accuracy assessment of NLCD 2006 land cover and impervious surface","volume":"130","author":"Wickham","year":"2013","journal-title":"Remote Sens. Environ."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"3907","DOI":"10.5194\/essd-13-3907-2021","article-title":"The 30 m annual land cover dataset and its dynamics in China from 1990 to 2019","volume":"13","author":"Yang","year":"2021","journal-title":"Earth Syst. Sci. Data"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Tyukavina, A., Potapov, P., Hansen, M.C., Pickens, A.H., Stehman, S.V., Turubanova, S., Parker, D., Zalles, V., Lima, A., and Kommareddy, I. (2022). Global Trends of Forest Loss Due to Fire from 2001 to 2019. Front. Remote Sens., 3.","DOI":"10.3389\/frsen.2022.825190"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Song, X.-P., Li, H., Potapov, P., and Hansen, M.C. (2022). Annual 30 m soybean yield mapping in Brazil using long-term satellite observations, climate data and machine learning. Agric. For. Meteorol., 326.","DOI":"10.1016\/j.agrformet.2022.109186"},{"key":"ref_31","unstructured":"Strahler, A.H., Boschetti, L., Foody, G.M., Friedl, M.A., Hansen, M.C., Herold, M., Mayaux, P., Morisette, J.T., Stehman, S.V., and Woodcock, C.E. (2006). Global Land Cover Validation: Recommendations for Evaluation and Accuracy Assessment of Global Land Cover Maps, Institute of Environmental Sustainability. Report of Institute of Environmental Sustainability; Joint Research Centre."},{"key":"ref_32","unstructured":"Cochran, W.G. (1977). Sampling Techniques, John Wiley & Sons. [3rd ed.]."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"331","DOI":"10.1016\/S0034-4257(98)00010-8","article-title":"Design and Analysis for Thematic Map Accuracy Assessment: Fundamental Principles","volume":"64","author":"Stehman","year":"1998","journal-title":"Remote Sens. Environ."},{"key":"ref_34","first-page":"727","article-title":"Statistical Rigor and Practical Utility in Thematic Map Accuracy Assessment","volume":"67","author":"Stehman","year":"2001","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"328","DOI":"10.1016\/j.rse.2016.12.026","article-title":"Thematic accuracy assessment of the 2011 National Land Cover Database (NLCD)","volume":"191","author":"Wickham","year":"2017","journal-title":"Remote Sens. Environ."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"2816","DOI":"10.1016\/j.rse.2010.07.001","article-title":"A scalable approach to mapping annual land cover at 250 m using MODIS time series data: A case study in the Dry Chaco ecoregion of South America","volume":"114","author":"Clark","year":"2010","journal-title":"Remote Sens. Environ."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"2972","DOI":"10.1109\/TGRS.2011.2122337","article-title":"Geolocation assessment of MERIS GlobCover orthorectified products","volume":"49","author":"Bicheron","year":"2011","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"111261","DOI":"10.1016\/j.rse.2019.111261","article-title":"Quality control and assessment of interpreter consistency of annual land cover reference data in an operational national monitoring program","volume":"238","author":"Pengra","year":"2020","journal-title":"Remote Sens. Environ."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1728","DOI":"10.1109\/TGRS.2006.864370","article-title":"Validation of the global land cover 2000 map","volume":"44","author":"Mayaux","year":"2006","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1038\/s41597-021-01105-4","article-title":"A crowdsourced global data set for validating built-up surface layers","volume":"9","author":"See","year":"2022","journal-title":"Sci. Data"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"202","DOI":"10.1016\/j.rse.2013.01.016","article-title":"Estimating area from an accuracy assessment error matrix","volume":"132","author":"Stehman","year":"2013","journal-title":"Remote Sens. Environ."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Xie, H., Wang, F., Gong, Y., Tong, X., Jin, Y., Zhao, A., Wei, C., Zhang, X., and Liao, S. (2022). Spatially Balanced Sampling for Validation of GlobeLand30 Using Landscape Pattern-Based Inclusion Probability. Sustainability, 14.","DOI":"10.3390\/su14052479"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Wang, Y., Zhang, J., Liu, D., Yang, W., and Zhang, W. (2018). Accuracy Assessment of GlobeLand30 2010 Land Cover over China Based on Geographically and Categorically Stratified Validation Sample Data. Remote Sens., 10.","DOI":"10.3390\/rs10081213"}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/15\/18\/4593\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T20:53:18Z","timestamp":1760129598000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/15\/18\/4593"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,18]]},"references-count":43,"journal-issue":{"issue":"18","published-online":{"date-parts":[[2023,9]]}},"alternative-id":["rs15184593"],"URL":"https:\/\/doi.org\/10.3390\/rs15184593","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,9,18]]}}}