{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,25]],"date-time":"2026-07-25T19:41:42Z","timestamp":1785008502492,"version":"3.55.0"},"reference-count":56,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2022,6,14]],"date-time":"2022-06-14T00:00:00Z","timestamp":1655164800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"University of Tabriz, International and Academic Cooperation Direction"},{"name":"Alexander Von Humboldt Foundation"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Classification is a very common image processing task. The accuracy of the classified map is typically assessed through a comparison with real-world situations or with available reference data to estimate the reliability of the classification results. Common accuracy assessment approaches are based on an error matrix and provide a measure for the overall accuracy. A frequently used index is the Kappa index. As the Kappa index has increasingly been criticized, various alternative measures have been investigated with minimal success in practice. In this article, we introduce a novel index that overcomes the limitations. Unlike Kappa, it is not sensitive to asymmetric distributions. The quantity and allocation disagreement index (QADI) index computes the degree of disagreement between the classification results and reference maps by counting wrongly labeled pixels as A and quantifying the difference in the pixel count for each class between the classified map and reference data as Q. These values are then used to determine a quantitative QADI index value, which indicates the value of disagreement and difference between a classification result and training data. It can also be used to generate a graph that indicates the degree to which each factor contributes to the disagreement. The efficiency of Kappa and QADI were compared in six use cases. The results indicate that the QADI index generates more reliable classification accuracy assessments than the traditional Kappa can do. We also developed a toolbox in a GIS software environment.<\/jats:p>","DOI":"10.3390\/s22124506","type":"journal-article","created":{"date-parts":[[2022,6,15]],"date-time":"2022-06-15T01:39:54Z","timestamp":1655257194000},"page":"4506","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":51,"title":["QADI as a New Method and Alternative to Kappa for Accuracy Assessment of Remote Sensing-Based Image Classification"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3367-2925","authenticated-orcid":false,"given":"Bakhtiar","family":"Feizizadeh","sequence":"first","affiliation":[{"name":"Department of Remote Sensing and GIS, University of Tabriz, Tabriz 516661647, Iran"},{"name":"GIScience Lab, Humboldt-Universit\u00e4t zu Berlin, 10117 Berlin, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sadrolah","family":"Darabi","sequence":"additional","affiliation":[{"name":"Department of Remote Sensing and GIS, University of Tabriz, Tabriz 516661647, Iran"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1860-8458","authenticated-orcid":false,"given":"Thomas","family":"Blaschke","sequence":"additional","affiliation":[{"name":"Department of Geoinformatics-Z-GIS, University of Salzburg, 5020 Salzburg, Austria"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tobia","family":"Lakes","sequence":"additional","affiliation":[{"name":"GIScience Lab, Humboldt-Universit\u00e4t zu Berlin, 10117 Berlin, Germany"},{"name":"IRI THESys, Humboldt-Universit\u00e4t zu Berlin, 10117 Berlin, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,6,14]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1109\/LGRS.2018.2867949","article-title":"A Framework for Remote Sensing Images Processing Using Deep Learning Techniques","volume":"16","author":"Cresson","year":"2018","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"832","DOI":"10.1080\/17538947.2019.1585976","article-title":"Big Earth data: Disruptive changes in Earth observation data management and analysis?","volume":"13","author":"Sudmanns","year":"2020","journal-title":"Int. J. Digit. Earth"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Chen, Z., Wagner, M., Das, J., Doe, R., and Cerveny, R. (2021). Data-Driven Approaches for Tornado Damage Estimation with Unpiloted Aerial Systems. Remote Sens., 13.","DOI":"10.3390\/rs13091669"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"741","DOI":"10.1080\/15481603.2017.1323377","article-title":"Deep learning in remote sensing scene classification: A data augmentation enhanced convolutional neural network framework","volume":"54","author":"Yu","year":"2017","journal-title":"GISci. Remote Sens."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"778","DOI":"10.1109\/LGRS.2017.2681128","article-title":"Deep Learning Classification of Land Cover and Crop Types Using Remote Sensing Data","volume":"14","author":"Kussul","year":"2017","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1016\/j.neunet.2017.07.017","article-title":"A patch-based convolutional neural network for remote sensing image classification","volume":"95","author":"Sharma","year":"2017","journal-title":"Neural Netw."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"2784","DOI":"10.1080\/01431161.2018.1433343","article-title":"Implementation of machine-learning classification in remote sensing: An applied review","volume":"39","author":"Maxwell","year":"2018","journal-title":"Int. J. Remote Sens."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1016\/j.isprsjprs.2017.03.001","article-title":"Disaster damage detection through synergistic use of deep learning and 3D point cloud features derived from very high resolution oblique aerial images, and multiple-kernel-learning","volume":"140","author":"Vetrivel","year":"2018","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Talukdar, S., Singha, P., Mahato, S., Pal, S., Liou, Y.-A., and Rahman, A. (2020). Land-Use Land-Cover Classification by Machine Learning Classifiers for Satellite Observations\u2014A Review. Remote Sens., 12.","DOI":"10.3390\/rs12071135"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"7779","DOI":"10.1080\/01431161.2020.1763511","article-title":"Dilated multi-scale cascade forest for satellite image classification","volume":"41","author":"Xia","year":"2020","journal-title":"Int. J. Remote Sens."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Najafi, P., Feizizadeh, B., and Navid, H. (2021). A Comparative Approach of Fuzzy Object Based Image Analysis and Machine Learning Techniques Which Are Applied to Crop Residue Cover Mapping by Using Sentinel-2 Satellite and UAV Imagery. Remote Sens., 13.","DOI":"10.3390\/rs13050937"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"837","DOI":"10.1007\/s12518-021-00393-0","article-title":"A comparative approach of support vector machine kernel functions for GIS-based landslide susceptibility mapping","volume":"13","author":"Kamran","year":"2021","journal-title":"Appl. Geomat."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1543","DOI":"10.1080\/15481603.2021.2000350","article-title":"A comparison of the integrated fuzzy object-based deep learning approach and three machine learning techniques for land use\/cover change monitoring and environmental impacts assessment","volume":"58","author":"Feizizadeh","year":"2021","journal-title":"GISci. Remote Sens."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"105585","DOI":"10.1016\/j.catena.2021.105585","article-title":"A deep learning convolutional neural network algorithm for detecting saline flow sources and mapping the environmental impacts of the Urmia Lake drought in Iran","volume":"207","author":"Feizizadeh","year":"2021","journal-title":"CATENA"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"146253","DOI":"10.1016\/j.scitotenv.2021.146253","article-title":"An automated deep learning convolutional neural network algorithm applied for soil salinity distribution mapping in Lake Urmia","volume":"778","author":"Garajeh","year":"2021","journal-title":"Iran. Sci. Total Environ."},{"key":"ref_16","unstructured":"Lillesand, T.M., and Kiefer, R.W. (2001). Remote Sensing and Image Interpretation, John Wiley and Sons. [4th ed.]."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1109\/LGRS.2017.2763979","article-title":"A Novel Approach of Fuzzy Dempster\u2013Shafer Theory for Spatial Uncertainty Analysis and Accuracy Assessment of Object-Based Image Classification","volume":"15","author":"Feizizadeh","year":"2018","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Maxwell, A., Warner, T., and Guill\u00e9n, L. (2021). Accuracy Assessment in Convolutional Neural Network-Based Deep Learning Remote Sensing Studies\u2014Part 2: Recommendations and Best Practices. Remote Sens., 13.","DOI":"10.3390\/rs13132591"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1016\/S0034-4257(01)00295-4","article-title":"Status of land cover classification accuracy assessment","volume":"80","author":"Foody","year":"2002","journal-title":"Remote Sens. Environ."},{"key":"ref_20","unstructured":"Piper, S.E. (1983). The evaluation of the spatial accuracy of computer classification. Machine Processing of Remotely Sensed Data Symposium, Purdue University."},{"key":"ref_21","first-page":"99","article-title":"The minimum accuracy value as an index of classification accuracy","volume":"51","author":"Aronoff","year":"1985","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_22","first-page":"223","article-title":"A coefficient of agreement as a measure of thematic classification accuracy","volume":"52","author":"Rosenfield","year":"1986","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_23","first-page":"246","article-title":"Statistical properties of five indices in assessing the accuracy of remotely sensed data using simple random sampling","volume":"2","author":"Kalkhan","year":"1995","journal-title":"Proc. ACSM\/ASPRS Annu. Conv. Expo."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"456","DOI":"10.3844\/ajeassp.2009.456.465","article-title":"A New Method for Accuracy Assessment of Sub-Pixel Classification Results","volume":"2","author":"Emami","year":"2009","journal-title":"Am. J. Eng. Appl. Sci."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"4621","DOI":"10.1109\/TGRS.2013.2283082","article-title":"A New Accuracy Assessment Method for One-Class Remote Sensing Classification","volume":"52","author":"Li","year":"2014","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Waldner, F. (2020). The T Index: Measuring the Reliability of Accuracy Estimates Obtained from Non-Probability Samples. Remote Sens., 12.","DOI":"10.3390\/rs12152483"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Radoux, J., and Bogaert, P. (2020). About the Pitfall of Erroneous Validation Data in the Estimation of Confusion Matrices. Remote Sens., 12.","DOI":"10.3390\/rs12244128"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Zhou, W., Troy, A., and Grove, J. (August, January 31). Measuring Urban Parcel Lawn Greenness by Using an Object-oriented Classification Approach. Proceedings of the 2006 IEEE International Symposium on Geoscience and Remote Sensing, Denver, CO, USA.","DOI":"10.1109\/IGARSS.2006.695"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"237","DOI":"10.1016\/j.rse.2012.09.005","article-title":"Spatial analysis of remote sensing image classification accuracy","volume":"127","author":"Comber","year":"2012","journal-title":"Remote Sens. Environ."},{"key":"ref_30","unstructured":"Foody, G.M. (2006, January 5\u20137). The evaluation and comparison of thematic maps derived from remote sensing. Proceedings of the 7th International Symposium on Spatial Accuracy Assessment in Natural Resources and Environmental Sciences, Lisbon, Portugal."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"111630","DOI":"10.1016\/j.rse.2019.111630","article-title":"Explaining the unsuitability of the kappa coefficient in the assessment and comparison of the accuracy of thematic maps obtained by image classification","volume":"239","author":"Foody","year":"2020","journal-title":"Remote Sens. Environ."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"240","DOI":"10.1016\/j.geomorph.2017.06.002","article-title":"Evaluating fuzzy operators of an object-based image analysis for detecting landslides and their changes","volume":"293","author":"Feizizadeh","year":"2017","journal-title":"Geomorphology"},{"key":"ref_33","first-page":"95","article-title":"The Kappa Statistic: A Second Look","volume":"30","author":"Glass","year":"2006","journal-title":"Comput. Linguist."},{"key":"ref_34","first-page":"751803","article-title":"Measurement of Interobserver Disagreement: Correction of Cohen\u2019s Kappa for Negative Values","volume":"2015","year":"2015","journal-title":"J. Probab. Stat."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1177\/001316446002000104","article-title":"A Coefficient of Agreement for Nominal Scales","volume":"20","author":"Cohen","year":"1960","journal-title":"Educ. Psychol. Meas."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"276","DOI":"10.11613\/BM.2012.031","article-title":"Interrater reliability: The kappa statistic","volume":"22","author":"McHugh","year":"2012","journal-title":"Biochem. Med."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"111199","DOI":"10.1016\/j.rse.2019.05.018","article-title":"Key issues in rigorous accuracy assessment of landcover products","volume":"231","author":"Stehman","year":"2019","journal-title":"Remote Sens. Environ."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"4407","DOI":"10.1080\/01431161.2011.552923","article-title":"Death to Kappa: Birth of quantity disagreement and allocation disagreement for accuracy assessment","volume":"32","author":"Pontius","year":"2011","journal-title":"Int. J. Remote Sens."},{"key":"ref_39","first-page":"C7","article-title":"Spatial accuracy assessment of object boundaries for object-based image analysis","volume":"38","author":"Albrecht","year":"2010","journal-title":"Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"687","DOI":"10.1177\/001316448104100307","article-title":"Coefficient kappa: Some uses, misuses, and alternatives","volume":"41","author":"Brennan","year":"1981","journal-title":"Educ. Psychol. Meas."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"159","DOI":"10.2307\/2529310","article-title":"The Measurement of Observer Agreement for Categorical Data","volume":"33","author":"Landis","year":"1977","journal-title":"Biometrics"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Fleiss, J.L., Levin, B., and Paik, M.C. (2003). Statistical Methods for Rates and Proportions, Wiley. [3rd ed.].","DOI":"10.1002\/0471445428"},{"key":"ref_43","unstructured":"Altman, D. (1999). Practical Statistics for Medical Research, CRC Press."},{"key":"ref_44","first-page":"411","article-title":"Reliability in Content Analysis: Some Common Misconceptions and Recommendations","volume":"30","author":"Krippendorff","year":"2004","journal-title":"Hum. Commun. Res."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"543","DOI":"10.1016\/0895-4356(90)90158-L","article-title":"High agreement but low kappa, the problems of two paradoxes","volume":"43","author":"Feinstein","year":"1989","journal-title":"J. Clin. Epidemiol."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"259","DOI":"10.1007\/s12517-017-3012-2","article-title":"An integrated object-based image analysis and CA-Markov model approach for modeling land use\/land cover trends in the Sarab plain","volume":"10","author":"Naboureh","year":"2017","journal-title":"Arab. J. Geosci."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Rousset, G., Despinoy, M., Schindler, K., and Mangeas, M. (2021). Assessment of Deep Learning Techniques for Land Use Land Cover Classification in Southern New Caledonia. Remote Sens., 13.","DOI":"10.3390\/rs13122257"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"43","DOI":"10.2478\/jengeo-2020-0005","article-title":"Machine Learning Techniques for Land Use\/Land Cover Classification of Medium Resolution Optical Satellite Imagery Focusing on Temporary Inundated Areas","volume":"13","author":"Tobak","year":"2020","journal-title":"J. Environ. Geogr."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"969","DOI":"10.1016\/0895-4356(88)90033-9","article-title":"Response Kappa and the concept of independent errors","volume":"41","author":"Thompson","year":"1988","journal-title":"J. Clin. Epidemiol."},{"key":"ref_50","first-page":"1","article-title":"Kappa Statistic is not Satisfactory for Assessing the Extent of Agreement Between Raters","volume":"1","author":"Gwet","year":"2002","journal-title":"Stat. Methods Inter-Rater Reliab. Assess."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1348\/000711006X126600","article-title":"Computing inter-rater reliability and its variance in the presence of high agreement","volume":"61","author":"Gwet","year":"2008","journal-title":"Br. J. Math. Stat. Psychol."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1016\/j.isprsjprs.2018.04.002","article-title":"A review of accuracy assessment for object-based image analysis: From per-pixel to per-polygon approaches","volume":"141","author":"Ye","year":"2018","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Feizizadeh, B., Omarzadeh, D., Kazemi Garajeh, M., Lakes, T., and Blaschke, T. (2021). Machine learning data-driven approaches for land use\/cover mapping and trend analysis using Google Earth Engine. J. Environ. Plan. Manag.","DOI":"10.1080\/09640568.2021.2001317"},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Feizizadeh, B., Lakes, T., Omarzadeh, D., Sharifi, A., Blaschke, T., and Karmizadeh, S.M. (2022). Scenario-based analysis of the impacts of lake drying on sustainable food production. Nat. Sci. Rep.","DOI":"10.21203\/rs.3.rs-934137\/v1"},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Zhang, L., Wu, J., Fan, Y., Gao, H., and Shao, Y. (2020). An efficient building extraction method from high spatial resolution remote sensing images based on improved mask R-CNN. Sensors, 20.","DOI":"10.3390\/s20051465"},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1016\/j.isprsjprs.2021.02.014","article-title":"Building outline delineation: From aerial images to polygons with an improved end-to-end learning framework","volume":"175","author":"Zhao","year":"2021","journal-title":"ISPRS J. Photogramm. Remote Sens."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/12\/4506\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:31:18Z","timestamp":1760139078000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/12\/4506"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,6,14]]},"references-count":56,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2022,6]]}},"alternative-id":["s22124506"],"URL":"https:\/\/doi.org\/10.3390\/s22124506","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,6,14]]}}}