{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,3]],"date-time":"2026-07-03T18:28:12Z","timestamp":1783103292372,"version":"3.54.6"},"reference-count":58,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2019,7,23]],"date-time":"2019-07-23T00:00:00Z","timestamp":1563840000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Key Research &amp; Development (R&amp;D) Plan of China","award":["2017YFB0503600"],"award-info":[{"award-number":["2017YFB0503600"]}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["41674100"],"award-info":[{"award-number":["41674100"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["IJGI"],"abstract":"<jats:p>Imbalanced learning is a methodological challenge in remote sensing communities, especially in complex areas where the spectral similarity exists between land covers. Obtaining high-confidence classification results for imbalanced class issues is highly important in practice. In this paper, extreme gradient boosting (XGB), a novel tree-based ensemble system, is employed to classify the land cover types in Very-high resolution (VHR) images with imbalanced training data. We introduce an extended margin criterion and disagreement performance to evaluate the efficiency of XGB in imbalanced learning situations and examine the effect of minority class spectral separability on model performance. The results suggest that the uncertainty of XGB associated with correct classification is stable. The average probability-based margin of correct classification provided by XGB is 0.82, which is about 46.30% higher than that by random forest (RF) method (0.56). Moreover, the performance uncertainty of XGB is insensitive to spectral separability after the sample imbalance reached a certain level (minority:majority &gt; 10:100). The impact of sample imbalance on the minority class is also related to its spectral separability, and XGB performs better than RF in terms of user accuracy for the minority class with imperfect separability. The disagreement components of XGB are better and more stable than RF with imbalanced samples, especially for complex areas with more types. In addition, appropriate sample imbalance helps to improve the trade-off between the recognition accuracy of XGB and the sample cost. According to our analysis, this margin-based uncertainty assessment and disagreement performance can help users identify the confidence level and error component in similar classification performance (overall, producer, and user accuracies).<\/jats:p>","DOI":"10.3390\/ijgi8070315","type":"journal-article","created":{"date-parts":[[2019,7,23]],"date-time":"2019-07-23T10:44:51Z","timestamp":1563878691000},"page":"315","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["Efficiency of Extreme Gradient Boosting for Imbalanced Land Cover Classification Using an Extended Margin and Disagreement Performance"],"prefix":"10.3390","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8442-6708","authenticated-orcid":false,"given":"Fei","family":"Sun","sequence":"first","affiliation":[{"name":"School of Geography and of Information Engineering, China University of Geosciences, No. 388 Lumo Road, Wuhan 430074, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5570-6391","authenticated-orcid":false,"given":"Run","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Geography and of Information Engineering, China University of Geosciences, No. 388 Lumo Road, Wuhan 430074, China"},{"name":"National Engineering Research Center of Geographic Information System, Wuhan 430074, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2387-5419","authenticated-orcid":false,"given":"Bo","family":"Wan","sequence":"additional","affiliation":[{"name":"School of Geography and of Information Engineering, China University of Geosciences, No. 388 Lumo Road, Wuhan 430074, China"},{"name":"National Engineering Research Center of Geographic Information System, Wuhan 430074, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yanjun","family":"Su","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Vegetation and Environmental Change, Institute of Botany, Chinese Academy of Sciences, Beijing 100093, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Qinghua","family":"Guo","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Vegetation and Environmental Change, Institute of Botany, Chinese Academy of Sciences, Beijing 100093, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Youxin","family":"Huang","sequence":"additional","affiliation":[{"name":"School of Geography and of Information Engineering, China University of Geosciences, No. 388 Lumo Road, Wuhan 430074, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xincai","family":"Wu","sequence":"additional","affiliation":[{"name":"School of Geography and of Information Engineering, China University of Geosciences, No. 388 Lumo Road, Wuhan 430074, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2019,7,23]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"155","DOI":"10.1016\/j.isprsjprs.2015.03.014","article-title":"Exploring issues of training data imbalance and mislabelling on random forest performance for large area land cover classification using the ensemble margin","volume":"105","author":"Mellor","year":"2015","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1016\/j.isprsjprs.2017.04.017","article-title":"Exploring diversity in ensemble classification: Applications in large area land cover mapping","volume":"129","author":"Mellor","year":"2017","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1016\/S0034-4257(01)00295-4","article-title":"Status of land cover classification accuracy assessment","volume":"80","author":"Foody","year":"2002","journal-title":"Remote Sens. Environ."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"175","DOI":"10.1016\/j.isprsjprs.2014.07.016","article-title":"Estimation of seismic building structural types using multi-sensor remote sensing and machine learning techniques","volume":"104","author":"Pelizari","year":"2015","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1201","DOI":"10.14358\/PERS.74.10.1201","article-title":"Mapping selective logging in mixed deciduous forest: A comparison of Machine Learning Algorithms","volume":"74","author":"Lippitt","year":"2008","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_6","first-page":"83","article-title":"Class imbalance in unsupervised change detection\u2014A diagnostic analysis from urban remote sensing","volume":"60","author":"Leichtle","year":"2017","journal-title":"Int. J. Appl. Earth Obs. Geoinf."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.rse.2006.03.004","article-title":"Training set size requirements for the classification of a specific class","volume":"104","author":"Foody","year":"2006","journal-title":"Remote Sens. Environ."},{"key":"ref_8","unstructured":"Foster, P. (2000, January 31). Machine Learning from Imbalanced Data Sets 101 (Extended Abstract). Proceedings of the AAAI\u20192000 Workshop on Imbalanced Data Sets, Austin, TX, USA."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"429","DOI":"10.3233\/IDA-2002-6504","article-title":"The class imbalance problem: A systematic study","volume":"6","author":"Japkowicz","year":"2002","journal-title":"Intell. Data Anal."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1263","DOI":"10.1109\/TKDE.2008.239","article-title":"Learning from Imbalanced Data","volume":"21","author":"He","year":"2009","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1007\/s13748-016-0094-0","article-title":"Learning from imbalanced data: Open challenges and future directions","volume":"5","author":"Krawczyk","year":"2016","journal-title":"Prog. Artif. Intell."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"220","DOI":"10.1016\/j.eswa.2016.12.035","article-title":"Learning from class-imbalanced data: Review of methods and applications","volume":"73","author":"Haixiang","year":"2017","journal-title":"Expert Syst. Appl."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"554","DOI":"10.1016\/j.asoc.2013.08.014","article-title":"Cost-sensitive decision tree ensembles for effective imbalanced classification","volume":"14","author":"Krawczyk","year":"2014","journal-title":"Appl. Soft Comput."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Ha, J., and Lee, J.-S. (2016, January 4\u20136). A New Under-Sampling Method Using Genetic Algorithm for Imbalanced Data Classification. Proceedings of the 10th International Conference on Ubiquitous Information Management and Communication, Danang, Vietnam.","DOI":"10.1145\/2857546.2857643"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"405","DOI":"10.1016\/j.eswa.2015.10.031","article-title":"Adaptive semi-unsupervised weighted oversampling (A-SUWO) for imbalanced datasets","volume":"46","author":"Nekooeimehr","year":"2016","journal-title":"Expert Syst. Appl."},{"key":"ref_16","first-page":"321","article-title":"SMOTE: Synthetic minority over-sampling technique","volume":"16","author":"Chawla","year":"2002","journal-title":"J. Artif. Int. Res."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1111\/j.0824-7935.2004.t01-1-00228.x","article-title":"A Multiple Resampling Method for Learning from Imbalanced Data Sets","volume":"20","author":"Andrew","year":"2004","journal-title":"Comput. Intell."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"3353","DOI":"10.1109\/TKDE.2016.2609424","article-title":"Online Bagging and Boosting for Imbalanced Data Streams","volume":"28","author":"Wang","year":"2016","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"714","DOI":"10.1016\/j.asoc.2015.08.060","article-title":"Evolutionary undersampling boosting for imbalanced classification of breast cancer malignancy","volume":"38","author":"Krawczyk","year":"2016","journal-title":"Appl. Soft Comput."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Hassan, A.K.I., and Abraham, A. (2016). Modeling Insurance Fraud Detection Using Imbalanced Data Classification. Advances in Nature and Biologically Inspired Computing, Springer.","DOI":"10.1007\/978-3-319-27400-3_11"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1016\/j.fss.2014.01.015","article-title":"Cost-sensitive linguistic fuzzy rule based classification systems under the MapReduce framework for imbalanced big data","volume":"258","author":"Herrera","year":"2015","journal-title":"Fuzzy Sets Syst."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1016\/j.neucom.2015.11.095","article-title":"Mixed-kernel based weighted extreme learning machine for inertial sensor based human activity recognition with imbalanced dataset","volume":"190","author":"Wu","year":"2016","journal-title":"Neurocomputing"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1323","DOI":"10.1016\/S0167-8655(97)00109-8","article-title":"Classification of imbalanced remote-sensing data by neural networks","volume":"18","author":"Bruzzone","year":"1997","journal-title":"Pattern Recognit. Lett."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Li, F., Li, S., Zhu, C., Lan, X., and Chang, H. (2017). Cost-Effective Class-Imbalance Aware CNN for Vehicle Localization and Categorization in High Resolution Aerial Images. Remote Sens., 9.","DOI":"10.3390\/rs9050494"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"781","DOI":"10.1109\/LGRS.2010.2048197","article-title":"Semisupervised Feature Selection for Unbalanced Sample Sets of VHR Images","volume":"7","author":"Chen","year":"2010","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Graves, J.S., Asner, P.G., Martin, E.R., Anderson, B.C., Colgan, S.M., Kalantari, L., and Bohlman, A.S. (2016). Tree Species Abundance Predictions in a Tropical Agricultural Landscape with a Supervised Classification Model and Imbalanced Data. Remote Sens., 8.","DOI":"10.3390\/rs8020161"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"935","DOI":"10.1016\/j.neucom.2015.04.120","article-title":"Study of the impact of resampling methods for contrast pattern based classifiers in imbalanced databases","volume":"175","year":"2016","journal-title":"Neurocomputing"},{"key":"ref_28","first-page":"1","article-title":"Landslide Susceptibility Assessment in Vietnam Using Support Vector Machines, Decision Tree, and Na\u00efve Bayes Models","volume":"2012","author":"Pradhan","year":"2012","journal-title":"Math. Probl. Eng."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"554","DOI":"10.1016\/S0034-4257(03)00132-9","article-title":"An assessment of the effectiveness of decision tree methods for land cover classification","volume":"86","author":"Pal","year":"2003","journal-title":"Remote Sens. Environ."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Chen, T., and Guestrin, C. (2016, January 13\u201317). XGBoost: A Scalable Tree Boosting System. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.","DOI":"10.1145\/2939672.2939785"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"102","DOI":"10.1016\/j.enconman.2018.02.087","article-title":"Comparison of Support Vector Machine and Extreme Gradient Boosting for predicting daily global solar radiation using temperature and precipitation in humid subtropical climates: A case study in China","volume":"164","author":"Fan","year":"2018","journal-title":"Energy Convers. Manag."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"304","DOI":"10.1016\/j.iref.2018.03.008","article-title":"Predicting failure in the U.S. banking sector: An extreme gradient boosting approach","volume":"61","author":"Carmona","year":"2019","journal-title":"Int. Rev. Econ. Finance"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"113","DOI":"10.1016\/j.ins.2013.07.007","article-title":"An insight into classification with imbalanced data: Empirical results and current trends on using data intrinsic characteristics","volume":"250","author":"Palade","year":"2013","journal-title":"Inf. Sci."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"7507","DOI":"10.1016\/j.eswa.2014.06.015","article-title":"Ensemble methods for multi-label classification","volume":"41","author":"Rokach","year":"2014","journal-title":"Expert Syst. Appl."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1016\/j.knosys.2018.05.037","article-title":"An Empirical Comparison on State-of-the-art Multi-class Imbalance Learning Algorithms and A New Diversified Ensemble Learning Scheme","volume":"158","author":"Bi","year":"2018","journal-title":"Knowl. Based Syst."},{"key":"ref_36","unstructured":"Schapire, R.E., Freund, Y., Barlett, P., and Lee, W.S. (1997, January 8\u201312). Boosting the margin: A new explanation for the effectiveness of voting methods. Proceedings of the 14th International Conference on Machine Learning (ICML \u201897), Nashville, TN, USA."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Panuju, D.R., Paull, D.J., and Trisasongko, B.H. (2019). Combining Binary and Post-Classification Change Analysis of Augmented ALOS Backscatter for Identifying Subtle Land Cover Changes. Remote Sens., 11.","DOI":"10.3390\/rs11010100"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"607","DOI":"10.1109\/LGRS.2018.2803259","article-title":"Very High Resolution Object-Based Land Use\u2013Land Cover Urban Classification Using Extreme Gradient Boosting","volume":"15","author":"Georganos","year":"2018","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Ustuner, M., and Balik Sanli, F. (2019). Polarimetric Target Decompositions and Light Gradient Boosting Machine for Crop Classification: A Comparative Evaluation. ISPRS Int. J. Geo-Inf., 8.","DOI":"10.3390\/ijgi8020097"},{"key":"ref_40","unstructured":"Branco, P., Torgo, L., and Ribeiro, R. (2015). A Survey of Predictive Mo delling under Imbalanced Distributions. CoRR. arXiv."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Maimon, O., and Rokach, L. (2010). Data Mining for Imbalanced Datasets: An Overview. Data Mining and Knowledge Discovery Handbook, Springer.","DOI":"10.1007\/978-0-387-09823-4"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"4407","DOI":"10.1080\/01431161.2011.552923","article-title":"Death to Kappa: Birth of quantity disagreement and allocation disagreement for accuracy assessment","volume":"32","author":"Pontius","year":"2011","journal-title":"Int. J. Remote Sens."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"7543","DOI":"10.1080\/2150704X.2014.969814","article-title":"Quantity, exchange, and shift components of difference in a square contingency table","volume":"35","author":"Pontius","year":"2014","journal-title":"Int. J. Remote Sens."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"595","DOI":"10.14358\/PERS.78.6.595","article-title":"A Framework for Supervised Image Classification with Incomplete Training Samples","volume":"78","author":"Guo","year":"2012","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_45","first-page":"106","article-title":"Estimating tree species diversity in the savannah using NDVI and woody canopy cover","volume":"66","author":"Madonsela","year":"2018","journal-title":"Int. J. Appl. Earth Obs. Geoinf."},{"key":"ref_46","unstructured":"McGarigal, K., Cushman, S.A., and Ene, E. (2019, May 01). FRAGSTATS v4: Spatial Pattern Analysis Program for Categorical and Continuous Maps. Available online: http:\/\/www.umass.edu\/landeco\/research\/fragstats\/fragstats.html."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"230","DOI":"10.1016\/S0034-4257(00)00169-3","article-title":"Classification and Change Detection Using Landsat TM Data: When and How to Correct Atmospheric Effects?","volume":"75","author":"Song","year":"2001","journal-title":"Remote Sens. Environ."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"610","DOI":"10.1109\/TSMC.1973.4309314","article-title":"Textural Features for Image Classification","volume":"3","author":"Haralick","year":"1973","journal-title":"IEEE Trans. Syst. Man Cybern."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"717","DOI":"10.1109\/TGRS.2010.2058578","article-title":"A Positive and Unlabeled Learning Algorithm for One-Class Classification of Remote-Sensing Data","volume":"49","author":"Li","year":"2011","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Richards, J.A. (1999). Remote Sensing Digital Image Analysis, Springer.","DOI":"10.1007\/978-3-662-03978-6"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1016\/j.biosystemseng.2018.04.011","article-title":"Pressure drop modelling in sand filters in micro-irrigation using gradient boosted regression trees","volume":"171","author":"Arbat","year":"2018","journal-title":"Biosyst. Eng."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Chen, L., Zhang, T., and Li, T. (2017, January 15\u201317). Gradient boosting model for unbalanced quantitative mass spectra quality assessment. Proceedings of the 2017 International Conference on Security, Pattern Analysis, and Cybernetics (SPAC), Shenzhen, China.","DOI":"10.1109\/SPAC.2017.8304311"},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1016\/j.eswa.2018.01.012","article-title":"A novel ensemble method for credit scoring: Adaption of different imbalance ratios","volume":"98","author":"He","year":"2018","journal-title":"Expert Syst. Appl."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1016\/j.isprsjprs.2016.01.011","article-title":"Random forest in remote sensing: A review of applications and future directions","volume":"114","author":"Belgiu","year":"2016","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1016\/j.isprsjprs.2011.11.002","article-title":"An assessment of the effectiveness of a random forest classifier for land-cover classification","volume":"67","author":"Ghimire","year":"2012","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1016\/j.neucom.2016.10.053","article-title":"Large cost-sensitive margin distribution machine for imbalanced data classification","volume":"224","author":"Cheng","year":"2017","journal-title":"Neurocomputing"},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1023\/A:1022859003006","article-title":"Measures of Diversity in Classifier Ensembles and Their Relationship with the Ensemble Accuracy","volume":"51","author":"Kuncheva","year":"2003","journal-title":"Mach. Learn."},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"112","DOI":"10.1016\/j.ins.2014.03.043","article-title":"On the use of MapReduce for imbalanced big data using Random Forest","volume":"285","author":"Herrera","year":"2014","journal-title":"Inf. Sci."}],"container-title":["ISPRS International Journal of Geo-Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2220-9964\/8\/7\/315\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:08:41Z","timestamp":1760188121000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2220-9964\/8\/7\/315"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,7,23]]},"references-count":58,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2019,7]]}},"alternative-id":["ijgi8070315"],"URL":"https:\/\/doi.org\/10.3390\/ijgi8070315","relation":{},"ISSN":["2220-9964"],"issn-type":[{"value":"2220-9964","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,7,23]]}}}