{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,20]],"date-time":"2026-04-20T17:11:18Z","timestamp":1776705078266,"version":"3.51.2"},"reference-count":29,"publisher":"Walter de Gruyter GmbH","issue":"1","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["The International Journal of Biostatistics"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Cohen\u2019s kappa coefficient, which was introduced in 1960, serves as the most widely employed coefficient to assess inter-observer agreement for categorical outcomes. However, the original kappa can only be applied to cross-sectional binary measurements and, therefore, cannot be applied in the practical situation when the observers evaluate the same subjects at repeated time intervals. This study summarizes six methods of assessing agreement of repeated binary outcomes under different assumptions and discusses under which condition we should use the most appropriate method in practice. These approaches are illustrated using data from the CDC anthrax vaccine adsorbed (AVA) human clinical trial comparing the agreement for two solicited adverse events after AVA between the 1\u20133 day in-clinic medical record and the patient\u2019s diary on the same day. We hope this article can inspire researchers to choose the most appropriate method to assess agreement for their own study with longitudinal binary data.<\/jats:p>","DOI":"10.1515\/ijb-2012-0001","type":"journal-article","created":{"date-parts":[[2013,7,29]],"date-time":"2013-07-29T16:07:01Z","timestamp":1375114021000},"page":"19-32","source":"Crossref","is-referenced-by-count":5,"title":["Assessing Agreement of Repeated Binary Measurements with an Application to the CDC\u2019s Anthrax Vaccine Clinical Trial"],"prefix":"10.1515","volume":"9","author":[{"given":"Yi","family":"Pan","sequence":"first","affiliation":[{"name":"Immunization Safety Office, Division of Healthcare Quality Promotion, National Center for Emerging and Zoonotic Infectious Diseases, Centers for Disease Control and Prevention, Atlanta, GA, USA; Logistics Health Inc, La Crosse, WI, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Charles E.","family":"Rose","sequence":"additional","affiliation":[{"name":"Division of Bacterial Diseases, National Center for Immunization and Respiratory Diseases, Centers for Disease Control and Prevention, Atlanta, GA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Michael","family":"Haber","sequence":"additional","affiliation":[{"name":"Department of Biostatistics and Bioinformatics, Emory University, Rollins School of Public Health, Atlanta, GA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yan","family":"Ma","sequence":"additional","affiliation":[{"name":"Biostatistics, Public Health Department, Hospital for Special Surgery, Weill Medical College of Cornell University, New York City, NY, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Josep L.","family":"Carrasco","sequence":"additional","affiliation":[{"name":"University of Barcelona, Barcelona, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Brock","family":"Stewart","sequence":"additional","affiliation":[{"name":"Immunization Safety Office, Division of Healthcare Quality Promotion, National Center for Emerging and Zoonotic Infectious Diseases, Centers for Disease Control and Prevention, Atlanta, GA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wendy A.","family":"Keitel","sequence":"additional","affiliation":[{"name":"Baylor College of Medicine, Houston, TX, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Harry","family":"Keyserling","sequence":"additional","affiliation":[{"name":"Emory University School of Medicine, Atlanta, GA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Robert M.","family":"Jacobson","sequence":"additional","affiliation":[{"name":"Department of Pediatric and Adolescent Medicine, Mayo Clinic, Rochester, MN, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Gregory","family":"Poland","sequence":"additional","affiliation":[{"name":"Mayo Clinic, Rochester, MN, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Michael M.","family":"McNeil","sequence":"additional","affiliation":[{"name":"Immunization Safety Office, Division of Healthcare Quality Promotion, National Center for Emerging and Zoonotic Infectious Diseases, Centers for Disease Control and Prevention; Centers for Disease Control and Prevention, 1600 Clifton Road NE, Atlanta, GA 30333"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"374","published-online":{"date-parts":[[2013,7,27]]},"reference":[{"key":"2025121208421223581_CIT0001_w2aab3b8d157b1b7b1ab2b2b1Aa","doi-asserted-by":"crossref","unstructured":"Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Meas 1960;20:37\u201346.10.1177\/001316446002000104","DOI":"10.1177\/001316446002000104"},{"key":"2025121208421223581_CIT0002_w2aab3b8d157b1b7b1ab2b2b2Aa","doi-asserted-by":"crossref","unstructured":"Cohen J. Weighted kappa: nominal scale agreement with provision for scaled disagreement or partial credit. Psychol Bull 1968;70:213\u201320.10.1037\/h0026256","DOI":"10.1037\/h0026256"},{"key":"2025121208421223581_CIT0003_w2aab3b8d157b1b7b1ab2b2b3Aa","doi-asserted-by":"crossref","unstructured":"Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986;i:307\u201310.10.1016\/S0140-6736(86)90837-8","DOI":"10.1016\/S0140-6736(86)90837-8"},{"key":"2025121208421223581_CIT0004_w2aab3b8d157b1b7b1ab2b2b4Aa","doi-asserted-by":"crossref","unstructured":"Bartko JJ. The intraclass correlation coefficient as a measure of reliability. Psychol Rep 1966;19:3C11.10.2466\/pr0.1966.19.1.3","DOI":"10.2466\/pr0.1966.19.1.3"},{"key":"2025121208421223581_CIT0005_w2aab3b8d157b1b7b1ab2b2b5Aa","doi-asserted-by":"crossref","unstructured":"Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull 1979;86:420C428.10.1037\/0033-2909.86.2.420","DOI":"10.1037\/\/0033-2909.86.2.420"},{"key":"2025121208421223581_CIT0006_w2aab3b8d157b1b7b1ab2b2b6Aa","doi-asserted-by":"crossref","unstructured":"Ebel RL. Estimation of the reliability of ratings. Psychometrika 1951;16:407\u201324.10.1007\/BF02288803","DOI":"10.1007\/BF02288803"},{"key":"2025121208421223581_CIT0007_w2aab3b8d157b1b7b1ab2b2b7Aa","unstructured":"Haggard EA. Intraclass correlation and the analysis of variance. New York: Dryden Press, 1958."},{"key":"2025121208421223581_CIT0008_w2aab3b8d157b1b7b1ab2b2b8Aa","doi-asserted-by":"crossref","unstructured":"Lin LI. A concordance correlation coefficient to evaluate reproducibility. Biometrics 1989;45:255\u201368.10.2307\/2532051","DOI":"10.2307\/2532051"},{"key":"2025121208421223581_CIT0009_w2aab3b8d157b1b7b1ab2b2b9Aa","doi-asserted-by":"crossref","unstructured":"Marano N, Plikaytis BD, Martin SW, et al. and Anthrax Vaccine Research Program Working Group. Effects of a reduced dose schedule and intramuscular administration of anthrax vaccine adsorbed on immunogenicity and safety at 7 months: a randomized trial. JAMA 2008;300:1532\u201343.10.1001\/jama.300.13.1532","DOI":"10.1001\/jama.300.13.1532"},{"key":"2025121208421223581_CIT0010_w2aab3b8d157b1b7b1ab2b2c10Aa","doi-asserted-by":"crossref","unstructured":"Coughlin SS, Pickle LW, Goodman MT, Wilkens LR. The logistic modeling of interobserver agreement. J Clin Epidemiol 1992;45:1237\u201341.10.1016\/0895-4356(92)90164-I","DOI":"10.1016\/0895-4356(92)90164-I"},{"key":"2025121208421223581_CIT0011_w2aab3b8d157b1b7b1ab2b2c11Aa","doi-asserted-by":"crossref","unstructured":"Lipsitz SR, Parzen M, Fitzmaurice GM, Klar N. A two-stage logistic regression model for analyzing inter-rater agreement. Psychometrika 2003;68:289\u201398.10.1007\/BF02294802","DOI":"10.1007\/BF02294802"},{"key":"2025121208421223581_CIT0012_w2aab3b8d157b1b7b1ab2b2c12Aa","doi-asserted-by":"crossref","unstructured":"Ma Y, Tang W, Feng C, Tu XM. Inference for kappas for longitudinal study data: applications to sexual health research. Biometrics 2008;64:781\u20139.10.1111\/j.1541-0420.2007.00934.x18047535","DOI":"10.1111\/j.1541-0420.2007.00934.x"},{"key":"2025121208421223581_CIT0013_w2aab3b8d157b1b7b1ab2b2c13Aa","doi-asserted-by":"crossref","unstructured":"Carrasco JL, King T, Chinchilli VM. The concordance correlation coefficient for repeated observations estimated by variance components. J Biopharm Stat 2009;19:90\u2013105.10.1080\/1054340080252789019127469","DOI":"10.1080\/10543400802527890"},{"key":"2025121208421223581_CIT0014_w2aab3b8d157b1b7b1ab2b2c14Aa","doi-asserted-by":"crossref","unstructured":"King TS, Chinchilli VM. A generalized concordance correlation coefficient for continuous and categorical data. Stat Med 2001;20:2131\u201347.10.1002\/sim.84511439426","DOI":"10.1002\/sim.845"},{"key":"2025121208421223581_CIT0015_w2aab3b8d157b1b7b1ab2b2c15Aa","unstructured":"Diggle PJ, Liang KY, Zeger SL. Analysis of longitudinal data. Oxford: Clarendon Press, 1994."},{"key":"2025121208421223581_CIT0016_w2aab3b8d157b1b7b1ab2b2c16Aa","doi-asserted-by":"crossref","unstructured":"Liang KY, Zeger SL. Longitudinal data analysis using generalized linear models. Biometrika 1986;73:13\u201322.10.1093\/biomet\/73.1.13","DOI":"10.1093\/biomet\/73.1.13"},{"key":"2025121208421223581_CIT0017_w2aab3b8d157b1b7b1ab2b2c17Aa","doi-asserted-by":"crossref","unstructured":"Liang KY, Zeger SL, Qaqish B. Multivariate regression analysis for categorical data. J R Stat Soc Ser B 1992;54:3\u201340.","DOI":"10.1111\/j.2517-6161.1992.tb01862.x"},{"key":"2025121208421223581_CIT0018_w2aab3b8d157b1b7b1ab2b2c18Aa","doi-asserted-by":"crossref","unstructured":"Muller R, Buttner P. A critical discussion of intraclass correlation coefficient. Stat Med 1984;13:2465\u201376.10.1002\/sim.47801323107701147","DOI":"10.1002\/sim.4780132310"},{"key":"2025121208421223581_CIT0019_w2aab3b8d157b1b7b1ab2b2c19Aa","doi-asserted-by":"crossref","unstructured":"Eliasziw M, Young, SL, Woodbury MG, Fryday-Field K. Statistical methodology for the concurrent assessment of interrater and intrarater reliability: Using goniometric measurements as an example. Phys Ther 1994;74:777\u201388.10.1093\/ptj\/74.8.7778047565","DOI":"10.1093\/ptj\/74.8.777"},{"key":"2025121208421223581_CIT0020_w2aab3b8d157b1b7b1ab2b2c20Aa","doi-asserted-by":"crossref","unstructured":"McGraw KO, Wong SP. Forming inferences about some intraclass correlation coefficients. Psychol Methods 1996;1:30\u201346.10.1037\/1082-989X.1.1.30","DOI":"10.1037\/\/1082-989X.1.1.30"},{"key":"2025121208421223581_CIT0021_w2aab3b8d157b1b7b1ab2b2c21Aa","doi-asserted-by":"crossref","unstructured":"Ridout MS, Demetrio CGB, Firth D. Estimating intraclass correlation for binary data. Biometrics 1999;55:137\u201348.10.1111\/j.0006-341X.1999.00137.x11318148","DOI":"10.1111\/j.0006-341X.1999.00137.x"},{"key":"2025121208421223581_CIT0022_w2aab3b8d157b1b7b1ab2b2c22Aa","doi-asserted-by":"crossref","unstructured":"King TS, Chinchilli VM, Wang KL, Carrasco JL. A class of repeated measures concordance correlation coefficients. J Biopharm Stat 2007;17:653\u201372.10.1080\/1054340070132945517613646","DOI":"10.1080\/10543400701329455"},{"key":"2025121208421223581_CIT0023_w2aab3b8d157b1b7b1ab2b2c23Aa","doi-asserted-by":"crossref","unstructured":"King TS, Chinchilli VM, Carrasco JL. A repeated observations concordance correlation coefficient. Stat Med 2007;26:3095\u2013113.10.1002\/sim.277817216594","DOI":"10.1002\/sim.2778"},{"key":"2025121208421223581_CIT0024_w2aab3b8d157b1b7b1ab2b2c24Aa","doi-asserted-by":"crossref","unstructured":"Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159\u201374.10.2307\/2529310","DOI":"10.2307\/2529310"},{"key":"2025121208421223581_CIT0025_w2aab3b8d157b1b7b1ab2b2c25Aa","doi-asserted-by":"crossref","unstructured":"Phillips KA, Milne RL, Buys S, Friedlander ML, Ward JH, McCredie MRE, et al. Agreement between self-reported breast cancer treatment and medical records in a population-based breast cancer family registry. J Clin Oncol 2005;23:4679\u201386.10.1200\/JCO.2005.03.00215851764","DOI":"10.1200\/JCO.2005.03.002"},{"key":"2025121208421223581_CIT0026_w2aab3b8d157b1b7b1ab2b2c26Aa","doi-asserted-by":"crossref","unstructured":"Phillip D, Lyngberg AC, Jensen R. Assessment of headache diagnosis: a comparative population study of a clinical interview with a diagnostic headache diary. Cephalalgia 2007;27:1.10.1111\/j.1468-2982.2007.01239.x17212676","DOI":"10.1111\/j.1468-2982.2007.01239.x"},{"key":"2025121208421223581_CIT0027_w2aab3b8d157b1b7b1ab2b2c27Aa","doi-asserted-by":"crossref","unstructured":"Tannenbaum C, Brouillette J, Corcos J. Rating improvements in urinary incontinence: do patients and their physicians agree? Age Aging 2008;37:379\u201383.10.1093\/ageing\/afn10818586834","DOI":"10.1093\/ageing\/afn108"},{"key":"2025121208421223581_CIT0028_w2aab3b8d157b1b7b1ab2b2c28Aa","doi-asserted-by":"crossref","unstructured":"Carrasco JL, Jover L, King T, Chinchilli VM. Comparison of concordance correlation coefficient estimating approaches with skewed data. J Biopharm Stat 2007;17:673\u201384.10.1080\/1054340070132946317613647","DOI":"10.1080\/10543400701329463"},{"key":"2025121208421223581_CIT0029_w2aab3b8d157b1b7b1ab2b2c29Aa","doi-asserted-by":"crossref","unstructured":"Zou G, Donner A. Confidence interval estimation of the intraclass correlation coefficient for binary outcome data. Biometrics 2004;60:807\u201311.10.1111\/j.0006-341X.2004.00232.x15339305","DOI":"10.1111\/j.0006-341X.2004.00232.x"}],"container-title":["ijb"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.degruyterbrill.com\/document\/doi\/10.1515\/ijb-2012-0001\/xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.degruyterbrill.com\/document\/doi\/10.1515\/ijb-2012-0001\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,12,12]],"date-time":"2025-12-12T10:45:22Z","timestamp":1765536322000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.degruyterbrill.com\/document\/doi\/10.1515\/ijb-2012-0001\/html"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2013,7,27]]},"references-count":29,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2013,7,27]]}},"alternative-id":["10.1515\/ijb-2012-0001"],"URL":"https:\/\/doi.org\/10.1515\/ijb-2012-0001","relation":{},"ISSN":["1557-4679","2194-573X"],"issn-type":[{"value":"1557-4679","type":"electronic"},{"value":"2194-573X","type":"print"}],"subject":[],"published":{"date-parts":[[2013,7,27]]}}}