{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,13]],"date-time":"2026-06-13T18:42:47Z","timestamp":1781376167495,"version":"3.54.1"},"reference-count":45,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2020,4,7]],"date-time":"2020-04-07T00:00:00Z","timestamp":1586217600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,4,7]],"date-time":"2020-04-07T00:00:00Z","timestamp":1586217600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001721","name":"University of Groningen","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100001721","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Adv Data Anal Classif"],"published-print":{"date-parts":[[2021,3]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Two types of nominal classifications are distinguished, namely regular nominal classifications and dichotomous-nominal classifications. The first type does not include an \u2018absence\u2019 category (for example, no disorder), whereas the second type does include an \u2018absence\u2019 category. Cohen\u2019s unweighted kappa can be used to quantify agreement between two regular nominal classifications with the same categories, but there are no coefficients for assessing agreement between two dichotomous-nominal classifications. Kappa coefficients for dichotomous-nominal classifications with identical categories are defined. All coefficients proposed belong to a one-parameter family. It is studied how the coefficients for dichotomous-nominal classifications are related and if the values of the coefficients depend on the number of categories. It turns out that the values of the new kappa coefficients can be strictly ordered in precisely two ways. The orderings suggest that the new coefficients are measuring the same thing, but to a different extent. If one accepts the use of magnitude guidelines, it is recommended to use stricter criteria for the new coefficients that tend to produce higher values.<\/jats:p>","DOI":"10.1007\/s11634-020-00394-8","type":"journal-article","created":{"date-parts":[[2020,4,7]],"date-time":"2020-04-07T21:12:40Z","timestamp":1586293960000},"page":"193-208","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":27,"title":["Kappa coefficients for dichotomous-nominal classifications"],"prefix":"10.1007","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7302-640X","authenticated-orcid":false,"given":"Matthijs J.","family":"Warrens","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2020,4,7]]},"reference":[{"key":"394_CR1","volume-title":"Practical statistics for medical research","author":"DG Altman","year":"1991","unstructured":"Altman DG (1991) Practical statistics for medical research. Chapman and Hall, London"},{"key":"394_CR2","volume-title":"Analysis of multivariate social science data","author":"DJ Bartholomew","year":"2011","unstructured":"Bartholomew DJ, Steele F, Moustaki I, Galbraith JI (2011) Analysis of multivariate social science data, 2nd edn. Chapman and Hall\/CRC, Boca Raton","edition":"2"},{"key":"394_CR3","doi-asserted-by":"crossref","first-page":"687","DOI":"10.1177\/001316448104100307","volume":"41","author":"RL Brennan","year":"1981","unstructured":"Brennan RL, Prediger DJ (1981) Coefficient kappa: some uses, misuses, and alternatives. Educ Psychol Meas 41:687\u2013699","journal-title":"Educ Psychol Meas"},{"key":"394_CR4","doi-asserted-by":"crossref","first-page":"673","DOI":"10.1080\/01688639208402855","volume":"14","author":"DV Cicchetti","year":"1992","unstructured":"Cicchetti DV, Volkmar F, Sparrow SS, Cohen D, Fermanian J, Rourke BP (1992) Assessing the reliability of clinical scales when the data have both nominal and ordinal features: proposed guidelines for neuropsychological assessments. J Clin Exp Neuropsychol 14:673\u2013686","journal-title":"J Clin Exp Neuropsychol"},{"key":"394_CR5","doi-asserted-by":"crossref","first-page":"213","DOI":"10.1177\/001316446002000104","volume":"20","author":"J Cohen","year":"1960","unstructured":"Cohen J (1960) A coefficient of agreement for nominal scales. Educ Psychol Meas 20:213\u2013220","journal-title":"Educ Psychol Meas"},{"key":"394_CR6","doi-asserted-by":"crossref","first-page":"213","DOI":"10.1037\/h0026256","volume":"70","author":"J Cohen","year":"1968","unstructured":"Cohen J (1968) Weighted kappa: nominal scale agreement with provision for scaled disagreement or partial credit. Psychol Bull 70:213\u2013220","journal-title":"Psychol Bull"},{"key":"394_CR7","doi-asserted-by":"crossref","first-page":"322","DOI":"10.1037\/0033-2909.88.2.322","volume":"88","author":"AJ Conger","year":"1980","unstructured":"Conger AJ (1980) Integration and generalization of kappas for multiple raters. Psychol Bull 88:322\u2013328","journal-title":"Psychol Bull"},{"key":"394_CR8","doi-asserted-by":"crossref","first-page":"1019","DOI":"10.1177\/0013164416663277","volume":"77","author":"AJ Conger","year":"2017","unstructured":"Conger AJ (2017) Kappa and rater accuracy: paradigms and parameters. Educ Psychol Meas 77:1019\u20131047","journal-title":"Educ Psychol Meas"},{"key":"394_CR9","doi-asserted-by":"crossref","first-page":"297","DOI":"10.1007\/BF02310555","volume":"16","author":"LJ Cronbach","year":"1951","unstructured":"Cronbach LJ (1951) Coefficient alpha and the internal structure of tests. Psychometrika 16:297\u2013334","journal-title":"Psychometrika"},{"key":"394_CR10","doi-asserted-by":"crossref","first-page":"558","DOI":"10.1177\/0013164418823249","volume":"79","author":"A De Raadt","year":"2019","unstructured":"De Raadt A, Warrens MJ, Bosker RJ, Kiers HAL (2019) Kappa coefficients for missing data. Educ Psychol Meas 79:558\u2013576","journal-title":"Educ Psychol Meas"},{"key":"394_CR11","doi-asserted-by":"crossref","first-page":"726","DOI":"10.1016\/j.neucom.2006.10.007","volume":"70","author":"W Dou","year":"2007","unstructured":"Dou W, Rena Y, Wu Q, Ruan S, Chen Y, Bloyet D, Constans J-M (2007) Fuzzy kappa for the agreement measure of fuzzy classifications. Neurocomputing 70:726\u2013734","journal-title":"Neurocomputing"},{"key":"394_CR12","volume-title":"Principles and practice of movement disorders","author":"S Fahn","year":"2011","unstructured":"Fahn S, Jankovic J, Hallett M (2011) Principles and practice of movement disorders. Elsevier Saunders, Edinburgh"},{"key":"394_CR13","doi-asserted-by":"crossref","first-page":"323","DOI":"10.1037\/h0028106","volume":"72","author":"JL Fleiss","year":"1969","unstructured":"Fleiss JL, Cohen J, Everitt BS (1969) Large sample standard errors of kappa and weighted kappa. Psychol Bull 72:323\u2013327","journal-title":"Psychol Bull"},{"key":"394_CR14","doi-asserted-by":"crossref","DOI":"10.1002\/0471445428","volume-title":"Statistical methods for rates and proportions","author":"JL Fleiss","year":"2003","unstructured":"Fleiss JL, Levin B, Paik MC (2003) Statistical methods for rates and proportions. Wiley, Hoboken"},{"key":"394_CR15","first-page":"732","volume":"49","author":"GD Goodman","year":"1954","unstructured":"Goodman GD, Kruskal WH (1954) Measures of association for cross classifications. J Am Stat Assoc 49:732\u2013764","journal-title":"J Am Stat Assoc"},{"key":"394_CR16","doi-asserted-by":"crossref","unstructured":"Gower JC, Warrens MJ (2017) Similarity, dissimilarity, and distance, measures of. Statistics Reference Online, Wiley StatsRef","DOI":"10.1002\/9781118445112.stat02470.pub2"},{"key":"394_CR17","volume-title":"Handbook of cluster analysis","author":"C Hennig","year":"2016","unstructured":"Hennig C, Meil\u01ce M, Murtagh F, Rocci R (2016) Handbook of cluster analysis. Chapman and Hall\/CRC, Boca Raton"},{"key":"394_CR18","doi-asserted-by":"crossref","first-page":"351","DOI":"10.1080\/13645579.2018.1547523","volume":"22","author":"R Hoekstra","year":"2019","unstructured":"Hoekstra R, Vugteveen J, Warrens MJ, Kruyen PM (2019) An empirical analysis of alleged misunderstandings of coefficient alpha. Int J Soc Res Methodol 22:351\u2013364","journal-title":"Int J Soc Res Methodol"},{"key":"394_CR19","doi-asserted-by":"crossref","first-page":"205","DOI":"10.1207\/S15328031US0203_03","volume":"2","author":"LM Hsu","year":"2003","unstructured":"Hsu LM, Field R (2003) Interrater agreement measures: comments on $$ {\\rm kappa}_{{\\rm n}}$$, Cohen\u2019s kappa, Scott\u2019s $$\\pi $$ and Aickin\u2019s $$\\alpha $$. Underst Stat 2:205\u2013219","journal-title":"Underst Stat"},{"key":"394_CR20","doi-asserted-by":"crossref","first-page":"193","DOI":"10.1007\/BF01908075","volume":"2","author":"L Hubert","year":"1985","unstructured":"Hubert L, Arabie P (1985) Comparing partitions. J Classif 2:193\u2013218","journal-title":"J Classif"},{"key":"394_CR21","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1111\/j.1469-8137.1912.tb05611.x","volume":"11","author":"P Jaccard","year":"1912","unstructured":"Jaccard P (1912) The distribution of the flora in the Alpine zone. New Phytol 11:37\u201350","journal-title":"New Phytol"},{"key":"394_CR22","doi-asserted-by":"crossref","first-page":"461","DOI":"10.1007\/BF02296208","volume":"44","author":"HC Kraemer","year":"1979","unstructured":"Kraemer HC (1979) Ramifications of a population model for $$\\kappa $$ as a coefficient of reliability. Psychometrika 44:461\u2013472","journal-title":"Psychometrika"},{"key":"394_CR23","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1148\/radiol.2282011860","volume":"288","author":"HL Kundel","year":"2003","unstructured":"Kundel HL, Polansky M (2003) Measurement of observer agreement. Radiology 288:303\u2013308","journal-title":"Radiology"},{"key":"394_CR24","doi-asserted-by":"crossref","first-page":"159","DOI":"10.2307\/2529310","volume":"33","author":"JR Landis","year":"1977","unstructured":"Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33:159\u2013174","journal-title":"Biometrics"},{"key":"394_CR25","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511663215","volume-title":"Assessment and diagnosis of personality disorders. The ICD-l0 international personality disorder examination (IPDE)","author":"AW Loranger","year":"1997","unstructured":"Loranger AW, Janca A, Sartorius N (1997) Assessment and diagnosis of personality disorders. The ICD-l0 international personality disorder examination (IPDE). Cambridge University Press, Cambridge"},{"key":"394_CR26","doi-asserted-by":"crossref","first-page":"161","DOI":"10.1093\/aje\/126.2.161","volume":"126","author":"M Maclure","year":"1987","unstructured":"Maclure M, Willett WC (1987) Misinterpretation and misuse of the kappa statistic. J Epidemiol 126:161\u2013169","journal-title":"J Epidemiol"},{"key":"394_CR27","first-page":"3769","volume":"46","author":"N Moradzadeh","year":"2017","unstructured":"Moradzadeh N, Ganjali M, Baghfalaki T (2017) Weighted kappa as a function of unweighted kappas. Commun Stat Simul Comput 46:3769\u20133780","journal-title":"Commun Stat Simul Comput"},{"key":"394_CR28","doi-asserted-by":"crossref","first-page":"397","DOI":"10.1038\/nnano.2014.38","volume":"9","author":"D Son","year":"2014","unstructured":"Son D, Lee J, Qiao S, Ghaffari R, Kim J, Lee JE, Kim D-H (2014) Multifunctional wearable devices for diagnosis and therapy of movement disorders. Nat Nanotechnol 9:397\u2013404","journal-title":"Nat Nanotechnol"},{"key":"394_CR29","doi-asserted-by":"crossref","first-page":"341","DOI":"10.1192\/bjp.125.4.341","volume":"125","author":"RL Spitzer","year":"1974","unstructured":"Spitzer RL, Fleiss JL (1974) A re-analysis of the reliability of psychiatric diagnosis. Br J Psychiatry 125:341\u2013347","journal-title":"Br J Psychiatry"},{"key":"394_CR30","doi-asserted-by":"crossref","first-page":"261","DOI":"10.1037\/met0000049","volume":"21","author":"D Steinley","year":"2016","unstructured":"Steinley D, Brusco MJ, Hubert L (2016) The variance of the adjusted Rand index. Psychol Methods 21:261\u2013272","journal-title":"Psychol Methods"},{"key":"394_CR31","doi-asserted-by":"crossref","first-page":"394","DOI":"10.1016\/j.learninstruc.2007.03.005","volume":"17","author":"J-W Strijbos","year":"2007","unstructured":"Strijbos J-W, Stahl G (2007) Methodological issues in developing a multi-dimensional coding procedure for small-group chat communication. Learn Instr 17:394\u2013404","journal-title":"Learn Instr"},{"key":"394_CR32","doi-asserted-by":"crossref","first-page":"399","DOI":"10.1007\/s11336-014-9439-4","volume":"81","author":"S Vanbelle","year":"2016","unstructured":"Vanbelle S (2016) A new interpretation of the weighted kappa coefficients. Psychometrika 81:399\u2013410","journal-title":"Psychometrika"},{"key":"394_CR33","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1016\/j.stamet.2008.06.001","volume":"6","author":"S Vanbelle","year":"2009","unstructured":"Vanbelle S, Albert A (2009) A note on the linearly weighted kappa coefficient for ordinal scales. Stat Methodol 6:157\u2013163","journal-title":"Stat Methodol"},{"key":"394_CR34","doi-asserted-by":"crossref","first-page":"3667","DOI":"10.1002\/sim.5424","volume":"31","author":"S Vanbelle","year":"2012","unstructured":"Vanbelle S, Mutsvari T, Declerck D, Lesaffre E (2012) Hierarchical modeling of agreement. Stat Med 31:3667\u20133680","journal-title":"Stat Med"},{"key":"394_CR35","doi-asserted-by":"crossref","first-page":"271","DOI":"10.1007\/s11634-010-0073-4","volume":"4","author":"MJ Warrens","year":"2010","unstructured":"Warrens MJ (2010) Inequalities between multi-rater kappas. Adv Data Anal Classif 4:271\u2013286","journal-title":"Adv Data Anal Classif"},{"key":"394_CR36","doi-asserted-by":"crossref","first-page":"473","DOI":"10.1016\/j.stamet.2011.06.002","volume":"8","author":"MJ Warrens","year":"2011","unstructured":"Warrens MJ (2011) Cohen\u2019s kappa is a weighted average. Stat Methodol 8:473\u2013484","journal-title":"Stat Methodol"},{"key":"394_CR37","doi-asserted-by":"crossref","first-page":"315","DOI":"10.1007\/s11336-012-9258-4","volume":"77","author":"MJ Warrens","year":"2012","unstructured":"Warrens MJ (2012) Some paradoxical results for the quadratically weighted kappa. Psychometrika 77:315\u2013323","journal-title":"Psychometrika"},{"key":"394_CR38","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1007\/s11634-013-0123-9","volume":"7","author":"MJ Warrens","year":"2013","unstructured":"Warrens MJ (2013) Cohen\u2019s weighted kappa with additive weights. Adv Data Anal Classif 7:41\u201355","journal-title":"Adv Data Anal Classif"},{"key":"394_CR39","doi-asserted-by":"crossref","first-page":"179","DOI":"10.1007\/s00357-014-9156-9","volume":"31","author":"MJ Warrens","year":"2014","unstructured":"Warrens MJ (2014) Corrected Zegers-ten Berge coefficients are special cases of Cohen\u2019s weighted kappa. J Classif 31:179\u2013193","journal-title":"J Classif"},{"key":"394_CR40","doi-asserted-by":"crossref","first-page":"197","DOI":"10.4172\/2161-0487.1000197","volume":"5","author":"MJ Warrens","year":"2015","unstructured":"Warrens MJ (2015) Five ways to look at Cohen\u2019s kappa. J Psychol Psychother 5:197","journal-title":"J Psychol Psychother"},{"key":"394_CR41","doi-asserted-by":"crossref","first-page":"385","DOI":"10.1016\/j.neucom.2016.02.038","volume":"194","author":"MJ Warrens","year":"2016","unstructured":"Warrens MJ (2016) Category kappas for agreement between fuzzy classifications. Neurocomputing 194:385\u2013388","journal-title":"Neurocomputing"},{"key":"394_CR42","doi-asserted-by":"crossref","first-page":"5240","DOI":"10.1080\/03610918.2016.1146765","volume":"46","author":"MJ Warrens","year":"2017","unstructured":"Warrens MJ (2017) Symmetric kappa as a function of unweighted kappas. Commun Stat Simul Comput 46:5240\u20135245","journal-title":"Commun Stat Simul Comput"},{"key":"394_CR43","doi-asserted-by":"crossref","first-page":"507","DOI":"10.1007\/s00357-016-9217-3","volume":"33","author":"MJ Warrens","year":"2016","unstructured":"Warrens MJ, Pratiwi BC (2016) Kappa coefficients for circular classifications. J Classif 33:507\u2013522","journal-title":"J Classif"},{"key":"394_CR44","doi-asserted-by":"crossref","first-page":"2612","DOI":"10.1002\/sim.6113","volume":"33","author":"Z Yang","year":"2014","unstructured":"Yang Z, Zhou M (2014) Kappa statistic for clustered matched-pair data. Stat Med 33:2612\u20132633","journal-title":"Stat Med"},{"key":"394_CR45","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.csda.2014.08.004","volume":"82","author":"Z Yang","year":"2015","unstructured":"Yang Z, Zhou M (2015) Weighted kappa statistic for clustered matched-pair ordinal data. Comput Stat Data Anal 82:1\u201318","journal-title":"Comput Stat Data Anal"}],"container-title":["Advances in Data Analysis and Classification"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11634-020-00394-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11634-020-00394-8\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11634-020-00394-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,4,10]],"date-time":"2021-04-10T06:26:17Z","timestamp":1618035977000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11634-020-00394-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,4,7]]},"references-count":45,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2021,3]]}},"alternative-id":["394"],"URL":"https:\/\/doi.org\/10.1007\/s11634-020-00394-8","relation":{},"ISSN":["1862-5347","1862-5355"],"issn-type":[{"value":"1862-5347","type":"print"},{"value":"1862-5355","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,4,7]]},"assertion":[{"value":"20 August 2019","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"9 March 2020","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"22 March 2020","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"7 April 2020","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}