{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,8]],"date-time":"2026-03-08T23:19:03Z","timestamp":1773011943776,"version":"3.50.1"},"reference-count":12,"publisher":"MIT Press","issue":"1","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Computational Linguistics"],"published-print":{"date-parts":[[2004,3]]},"abstract":"<jats:p> In recent years, the kappa coefficient of agreement has become the de facto standard for evaluating intercoder agreement for tagging tasks. In this squib, we highlight issues that affect \u03ba and that the community has largely neglected. First, we discuss the assumptions underlying different computations of the expected agreement component of \u03ba. Second, we discuss how prevalence and bias affect the \u03ba measure. <\/jats:p>","DOI":"10.1162\/089120104773633402","type":"journal-article","created":{"date-parts":[[2004,4,13]],"date-time":"2004-04-13T22:47:45Z","timestamp":1081896465000},"page":"95-101","source":"Crossref","is-referenced-by-count":255,"title":["The Kappa Statistic: A Second Look"],"prefix":"10.1162","volume":"30","author":[{"given":"Barbara Di","family":"Eugenio","sequence":"first","affiliation":[{"name":"University of Illinois at Chicago, Computer Science, 1120 SEO (M\/C 152), 851 South Morgan Street, Chicago, IL 60607."}]},{"given":"Michael","family":"Glass","sequence":"additional","affiliation":[{"name":"Valparaiso University, Mathematics and Computer Science, 116 Gellerson Hall, Valparaiso, IN 46383."}]}],"member":"281","reference":[{"key":"p_2","doi-asserted-by":"publisher","DOI":"10.1097\/00005053-197611000-00003"},{"key":"p_3","doi-asserted-by":"publisher","DOI":"10.1001\/jama.268.18.2513"},{"key":"p_4","doi-asserted-by":"publisher","DOI":"10.1016\/0895-4356(93)90018-V"},{"issue":"2","key":"p_5","first-page":"249","volume":"22","author":"Carletta Jean","year":"1996","journal-title":"Computational Linguistics"},{"issue":"1","key":"p_6","first-page":"13","volume":"23","author":"Carletta Jean","year":"1997","journal-title":"Computational Lingustics"},{"key":"p_7","doi-asserted-by":"publisher","DOI":"10.1016\/0895-4356(90)90159-M"},{"key":"p_8","doi-asserted-by":"publisher","DOI":"10.1177\/001316446002000104"},{"key":"p_10","doi-asserted-by":"publisher","DOI":"10.1006\/ijhc.2000.0428"},{"key":"p_11","doi-asserted-by":"publisher","DOI":"10.1037\/h0031619"},{"key":"p_12","doi-asserted-by":"publisher","DOI":"10.1001\/jama.268.18.2513"},{"key":"p_13","doi-asserted-by":"publisher","DOI":"10.1001\/archpsyc.1981.01780290042004"},{"key":"p_16","doi-asserted-by":"publisher","DOI":"10.1086\/266577"}],"container-title":["Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mitpressjournals.org\/doi\/pdf\/10.1162\/089120104773633402","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,3,12]],"date-time":"2021-03-12T21:42:20Z","timestamp":1615585340000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/coli\/article\/30\/1\/95-101\/1835"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2004,3]]},"references-count":12,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2004,3]]}},"alternative-id":["10.1162\/089120104773633402"],"URL":"https:\/\/doi.org\/10.1162\/089120104773633402","relation":{},"ISSN":["0891-2017","1530-9312"],"issn-type":[{"value":"0891-2017","type":"print"},{"value":"1530-9312","type":"electronic"}],"subject":[],"published":{"date-parts":[[2004,3]]}}}