{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,27]],"date-time":"2026-03-27T20:08:44Z","timestamp":1774642124300,"version":"3.50.1"},"reference-count":53,"publisher":"MDPI AG","issue":"15","license":[{"start":{"date-parts":[[2024,7,30]],"date-time":"2024-07-30T00:00:00Z","timestamp":1722297600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Texas Comptroller\u2019s Office"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>In the United States, several land use and land cover (LULC) data sets are available based on satellite data, but these data sets often fail to accurately represent features on the ground. Alternatively, detailed mapping of heterogeneous landscapes for informed decision-making is possible using high spatial resolution orthoimagery from the National Agricultural Imagery Program (NAIP). However, large-area mapping at this resolution remains challenging due to radiometric differences among scenes, landscape heterogeneity, and computational limitations. Various machine learning (ML) techniques have shown promise in improving LULC maps. The primary purposes of this study were to evaluate bagging (Random Forest, RF), boosting (Gradient Boosting Machines [GBM] and extreme gradient boosting [XGB]), and stacking ensemble ML models. We used these techniques on a time series of Sentinel 2A data and NAIP orthoimagery to create a LULC map of a portion of Irion and Tom Green counties in Texas (USA). We created several spectral indices, structural variables, and geometry-based variables, reducing the dimensionality of features generated on Sentinel and NAIP data. We then compared accuracy based on random cross-validation without accounting for spatial autocorrelation and target-oriented cross-validation accounting for spatial structures of the training data set. Comparison of random and target-oriented cross-validation results showed that autocorrelation in the training data offered overestimation ranging from 2% to 3.5%. The XGB-boosted stacking ensemble on-base learners (RF, XGB, and GBM) improved model performance over individual base learners. We show that meta-learners are just as sensitive to overfitting as base models, as these algorithms are not designed to account for spatial information. Finally, we show that the fusion of Sentinel 2A data with NAIP data improves land use\/land cover classification using geographic object-based image analysis.<\/jats:p>","DOI":"10.3390\/rs16152778","type":"journal-article","created":{"date-parts":[[2024,7,30]],"date-time":"2024-07-30T08:51:42Z","timestamp":1722329502000},"page":"2778","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Ensemble Machine Learning on the Fusion of Sentinel Time Series Imagery with High-Resolution Orthoimagery for Improved Land Use\/Land Cover Mapping"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4167-0206","authenticated-orcid":false,"given":"Mukti Ram","family":"Subedi","sequence":"first","affiliation":[{"name":"Department of Natural Resources Management, Texas Tech University, Lubbock, TX 79409, USA"},{"name":"Warnell School of Forestry and Natural Resources, University of Georgia, Athens, GA 30602, USA"}]},{"given":"Carlos","family":"Portillo-Quintero","sequence":"additional","affiliation":[{"name":"Department of Natural Resources Management, Texas Tech University, Lubbock, TX 79409, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1790-3057","authenticated-orcid":false,"given":"Nancy E.","family":"McIntyre","sequence":"additional","affiliation":[{"name":"Department of Biological Sciences, Texas Tech University, Lubbock, TX 79409, USA"}]},{"given":"Samantha S.","family":"Kahl","sequence":"additional","affiliation":[{"name":"Department of Biology, Blackburn College, Carlinville, IL 62626, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4005-1019","authenticated-orcid":false,"given":"Robert D.","family":"Cox","sequence":"additional","affiliation":[{"name":"Department of Natural Resources Management, Texas Tech University, Lubbock, TX 79409, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0245-2513","authenticated-orcid":false,"given":"Gad","family":"Perry","sequence":"additional","affiliation":[{"name":"Department of Natural Resources Management, Texas Tech University, Lubbock, TX 79409, USA"},{"name":"Department of Environmental Science and Policy, George Mason University, 4400 University Dr., Fairfax, VA 22030, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5514-0321","authenticated-orcid":false,"given":"Xiaopeng","family":"Song","sequence":"additional","affiliation":[{"name":"Department of Geographical Sciences, University of Maryland, College Park, MD 20742, USA"}]}],"member":"1968","published-online":{"date-parts":[[2024,7,30]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"2542","DOI":"10.1080\/01431161.2018.1528400","article-title":"Evaluating Multiple Classifier System for the Reduction of Salt-and-Pepper Noise in the Classification of Very-High-Resolution Satellite Images","volume":"40","author":"Hirayama","year":"2019","journal-title":"Int. J. Remote Sens."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"301","DOI":"10.1080\/15481603.2014.912874","article-title":"Comparison of NAIP Orthophotography and Rapideye Satellite Imagery for Mapping of Mining and Mine Reclamation","volume":"51","author":"Maxwell","year":"2014","journal-title":"GISci. Remote Sens."},{"key":"ref_3","first-page":"345","article-title":"Completion of the 2011 National Land Cover Database for the Conterminous United States-Representing a Decade of Land Cover Change Information","volume":"81","author":"Homer","year":"2015","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_4","first-page":"858","article-title":"Completion of the 2006 National Land Cover Database for the Conterminous United States","volume":"77","author":"Fry","year":"2011","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Blaschke, T., Lang, S., and Hay, G. (2008). Geographic Object-Based Image Analysis (GEOBIA): A New Name for a New Discipline. Object-Based Image Analysis: Spatial Concepts for Knowledge-Driven Remote Sensing Applications, Springer.","DOI":"10.1007\/978-3-540-77058-9"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"112","DOI":"10.1080\/2150704X.2014.882526","article-title":"High-Resolution Landcover Classification Using Random Forest","volume":"5","author":"Hayes","year":"2014","journal-title":"Remote Sens. Lett."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"613","DOI":"10.14358\/PERS.79.7.613","article-title":"The Effects of Data Selection and Thematic Detail on the Accuracy of High Spatial Resolution Wetland Classifications","volume":"79","author":"Knight","year":"2013","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"14096","DOI":"10.1038\/s41598-020-69743-z","article-title":"Evaluating the Integrity of Forested Riparian Buffers over a Large Area Using LiDAR Data and Google Earth Engine","volume":"10","author":"Zurqani","year":"2020","journal-title":"Sci. Rep."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"547","DOI":"10.14358\/PERS.22-00123R2","article-title":"Leveraging NAIP Imagery for Accurate Large-Area Land Use\/Land Cover Mapping: A Case Study in Central Texas","volume":"89","author":"Subedi","year":"2023","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"11372","DOI":"10.3390\/rs61111372","article-title":"Object-Based Land-Cover Mapping with High Resolution Aerial Photography at a County Scale in Midwestern USA","volume":"6","author":"Li","year":"2014","journal-title":"Remote Sens."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"54","DOI":"10.1007\/s40808-016-0108-8","article-title":"A Support Vector Machine Object Based Image Analysis Approach on Urban Green Space Extraction Using Pleiades-1A Imagery","volume":"2","author":"Sulma","year":"2016","journal-title":"Model. Earth Syst. Environ."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Blaschke, T., Lang, S., and Hay, G. (2008). Support Vector Machine Classification for Object-Based Image Analysis. Object-Based Image Analysis: Spatial Concepts for Knowledge-Driven Remote Sensing Applications, Springer.","DOI":"10.1007\/978-3-540-77058-9"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"443","DOI":"10.1080\/10106049.2016.1265595","article-title":"An Object-Based Approach for Mapping Forest Structural Types Based on Low-Density LiDAR and Multispectral Imagery","volume":"33","author":"Ruiz","year":"2018","journal-title":"Geocarto Int."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1080\/10095020.2017.1399674","article-title":"Object-Based Classification of Hyperspectral Data Using Random Forest Algorithm","volume":"21","author":"Amini","year":"2018","journal-title":"Geo-Spat. Inf. Sci."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"43","DOI":"10.2478\/jengeo-2020-0005","article-title":"Machine Learning Techniques for Land Use\/Land Cover Classification of Medium Resolution Optical Satellite Imagery Focusing on Temporary Inundated Areas","volume":"13","author":"Tobak","year":"2020","journal-title":"J. Environ. Geogr."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1145","DOI":"10.1016\/j.rse.2010.12.017","article-title":"Per-Pixel vs. Object-Based Classification of Urban Land Cover Extraction Using High Spatial Resolution Imagery","volume":"115","author":"Myint","year":"2011","journal-title":"Remote Sens. Environ."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"799","DOI":"10.14358\/PERS.72.7.799","article-title":"Object-Based Detailed Vegetation Classification with Airborne High Spatial Resolution Remote Sensing Imagery","volume":"72","author":"Yu","year":"2006","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"111716","DOI":"10.1016\/j.rse.2020.111716","article-title":"Deep Learning in Environmental Remote Sensing: Achievements and Challenges","volume":"241","author":"Yuan","year":"2020","journal-title":"Remote Sens. Environ."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Taghizadeh-Mehrjardi, R., Schmidt, K., Amirian-Chakan, A., Rentschler, T., Zeraatpisheh, M., Sarmadian, F., Valavi, R., Davatgar, N., Behrens, T., and Scholten, T. (2020). Improving the Spatial Prediction of Soil Organic Carbon Content in Two Contrasting Climatic Regions by Stacking Machine Learning Models and Rescanning Covariate Space. Remote Sens., 12.","DOI":"10.3390\/rs12071095"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"106485","DOI":"10.1016\/j.catena.2022.106485","article-title":"Comparison of Bagging, Boosting and Stacking Algorithms for Surface Soil Moisture Mapping Using Optical-Thermal-Microwave Remote Sensing Synergies","volume":"217","author":"Das","year":"2022","journal-title":"Catena"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Jafarzadeh, H., Mahdianpari, M., Gill, E., Mohammadimanesh, F., and Homayouni, S. (2021). Bagging and Boosting Ensemble Classifiers for Classification of Multispectral, Hyperspectral and PolSAR Data: A Comparative Evaluation. Remote Sens., 13.","DOI":"10.3390\/rs13214405"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Wu, X., and Wang, J. (2023). Application of Bagging, Boosting and Stacking Ensemble and EasyEnsemble Methods for Landslide Susceptibility Mapping in the Three Gorges Reservoir Area of China. Int. J. Environ. Res. Public Health, 20.","DOI":"10.3390\/ijerph20064977"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"2217","DOI":"10.1109\/JSTARS.2019.2918242","article-title":"Eurosat: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification","volume":"12","author":"Helber","year":"2019","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1016\/j.isprsjprs.2020.12.010","article-title":"Review on Convolutional Neural Networks(CNN) in Vegetation Remote Sensing","volume":"173","author":"Kattenborn","year":"2021","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"601","DOI":"10.1034\/j.1600-0587.2002.250508.x","article-title":"The Consequences of Spatial Structure for the Design and Analysis of Ecological Field Surveys","volume":"25","author":"Legendre","year":"2002","journal-title":"Ecography"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"297","DOI":"10.1111\/j.1538-4632.2008.00727.x","article-title":"A History of the Concept of Spatial Autocorrelation: A Geographer\u2019s Perspective","volume":"40","author":"Getis","year":"2008","journal-title":"Geogr. Anal."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"111199","DOI":"10.1016\/j.rse.2019.05.018","article-title":"Key Issues in Rigorous Accuracy Assessment of Land Cover Products","volume":"231","author":"Stehman","year":"2019","journal-title":"Remote Sens. Environ."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"913","DOI":"10.1111\/ecog.02881","article-title":"Cross-Validation Strategies for Data with Temporal, Spatial, Hierarchical, or Phylogenetic Structure","volume":"40","author":"Roberts","year":"2017","journal-title":"Ecography"},{"key":"ref_29","unstructured":"Griffith, G.E., Bryce, S., Omernik, J., and Rogers, A. (2004). Ecoregions of Texas. U.S. Environmental Protection Agency."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1016\/j.rse.2011.11.026","article-title":"Sentinel-2: ESA\u2019s Optical High-Resolution Mission for GMES Operational Services","volume":"120","author":"Drusch","year":"2012","journal-title":"Remote Sens. Environ."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"12242","DOI":"10.3390\/rs70912242","article-title":"SPOT-4(Take 5): Simulation of Sentinel-2 Time Series on 45 Large Sites","volume":"7","author":"Hagolle","year":"2015","journal-title":"Remote Sens."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"2627","DOI":"10.1080\/01431160120769","article-title":"Texture Analysis of IKONOS Panchromatic Data for Douglas-Fir Forest Age Class Separability in British Columbia","volume":"22","author":"Franklin","year":"2001","journal-title":"Int. J. Remote Sens."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"610","DOI":"10.1109\/TSMC.1973.4309314","article-title":"Textural Features for Image Classification","volume":"6","author":"Haralick","year":"1973","journal-title":"IEEE Trans. Syst. Man Cybern."},{"key":"ref_34","unstructured":"Legendre, P., and Legendre, L. (2012). Numerical Ecology, Elsevier. [3rd ed.]."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1016\/j.rse.2010.06.016","article-title":"An Infrared Desert Dust Index for the Along-Track Scanning Radiometers","volume":"116","author":"Good","year":"2012","journal-title":"Remote Sens. Environ."},{"key":"ref_36","unstructured":"(2020). ECognition Developer, Trimble. version 9."},{"key":"ref_37","first-page":"2825","article-title":"Scikit-Learn: Machine Learning in Python","volume":"12","author":"Pedregosa","year":"2011","journal-title":"J. Mach. Learn. Res."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Kuhn, M., and Johnson, K. (2013). Classification Trees and Rule-Based Models BT. Applied Predictive Modeling, Springer.","DOI":"10.1007\/978-1-4614-6849-3_14"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random Forests","volume":"45","author":"Breiman","year":"2001","journal-title":"Mach. Learn."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"8859","DOI":"10.1080\/01431161.2018.1500071","article-title":"Integrating LiDAR and High-Resolution Imagery for Object-Based Mapping of Forest Habitats in a Heterogeneous Temperate Forest Landscape","volume":"39","author":"Latifi","year":"2018","journal-title":"Int. J. Remote Sens."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1016\/j.isprsjprs.2010.08.007","article-title":"Relevance of Airborne Lidar and Multispectral Image Data for Urban Scene Classification Using Random Forests","volume":"66","author":"Guo","year":"2011","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"1189","DOI":"10.1214\/aos\/1013203451","article-title":"Greedy function approximation: A gradient boosting machine","volume":"29","author":"Friedman","year":"2001","journal-title":"Ann. Stat."},{"key":"ref_43","unstructured":"Feng, J., Xu, Y.-X., Jiang, Y., and Zhou, Z.-H. (2020). Soft Gradient Boosting Machine. arXiv."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Chen, T., and Guestrin, C. (2016, January 13). XGBoost: A Scalable Tree Boosting System. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.","DOI":"10.1145\/2939672.2939785"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1016\/S0893-6080(05)80023-1","article-title":"Stacked Generalization","volume":"5","author":"Wolpert","year":"1992","journal-title":"Neural Netw."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1016\/0034-4257(91)90048-B","article-title":"A Review of Assessing the Accuracy of Classifications of Remotely Sensed Data","volume":"37","author":"Congalton","year":"1991","journal-title":"Remote Sens. Environ."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"442","DOI":"10.1016\/0005-2795(75)90109-9","article-title":"Comparison of the Predicted and Observed Secondary Structure of T4 Phage Lysozyme","volume":"405","author":"Matthews","year":"1975","journal-title":"Biochim. Biophys. Acta-Protein Struct."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"641","DOI":"10.1007\/s10346-019-01286-5","article-title":"Improved Landslide Assessment Using Support Vector Machine with Bagging, Boosting, and Stacking Ensemble Machine Learning Framework in a Mountainous Watershed, Japan","volume":"17","author":"Dou","year":"2020","journal-title":"Landslides"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.envsoft.2017.12.001","article-title":"Improving Performance of Spatio-Temporal Machine Learning Models Using Forward Feature Selection and Target-Oriented Validation","volume":"101","author":"Meyer","year":"2018","journal-title":"Environ. Model. Softw."},{"key":"ref_50","first-page":"593","article-title":"A Comparison of Sampling Schemes Used in Generating Error Matrices for Assessing the Accuracy of Maps Generated from Remotely Sensed Data","volume":"54","author":"Congalton","year":"1998","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"109692","DOI":"10.1016\/j.ecolmodel.2021.109692","article-title":"Spatial Cross-Validation Is Not the Right Way to Evaluate Map Accuracy","volume":"457","author":"Wadoux","year":"2021","journal-title":"Ecol. Model."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"2715","DOI":"10.1007\/s10994-021-05972-1","article-title":"Spatial Dependence between Training and Test Sets: Another Pitfall of Classification Accuracy Assessment in Remote Sensing","volume":"111","author":"Karasiak","year":"2021","journal-title":"Mach. Learn."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"5321","DOI":"10.1080\/01431161.2010.498841","article-title":"Impact of Reference Datasets and Autocorrelation on Classification Accuracy","volume":"32","author":"Mannel","year":"2011","journal-title":"Int. J. Remote Sens."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/16\/15\/2778\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T15:26:17Z","timestamp":1760109977000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/16\/15\/2778"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,30]]},"references-count":53,"journal-issue":{"issue":"15","published-online":{"date-parts":[[2024,8]]}},"alternative-id":["rs16152778"],"URL":"https:\/\/doi.org\/10.3390\/rs16152778","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,7,30]]}}}