{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,22]],"date-time":"2026-09-22T08:45:26Z","timestamp":1790066726561,"version":"4.0.1"},"reference-count":19,"publisher":"SAGE Publications","issue":"6","license":[{"start":{"date-parts":[[2016,7,11]],"date-time":"2016-07-11T00:00:00Z","timestamp":1468195200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Stat Methods Med Res"],"published-print":{"date-parts":[[2016,12]]},"abstract":"<jats:p>Clinical core laboratories, such as Echocardiography core laboratories, are increasingly used in clinical studies with imaging outcomes as primary, secondary, or surrogate endpoints. While many factors contribute to the quality of measurements of imaging variables, an essential step in ensuring the value of imaging data includes formal assessment and control of reproducibility via intra-observer and inter-observer reliability. There are many different agreement\/reliability indices in the literature. However, different indices may lead to different conclusions and it is not clear which index is the preferred choice as an overall indication of data quality and a tool for providing guidance on improving quality and reliability in a core lab setting. In this paper, we pre-specify the desirable characteristics of an agreement index for assessing and improving reproducibility in a core lab setting; we compare existing agreement indices in terms of these characteristics to choose a preferred index. We conclude that, among the existing indices reviewed, the coverage probability for assessing agreement is the preferred agreement index on the basis of computational simplicity, its ability for rapid identification of discordant measurements to provide guidance for review and retraining, and its consistent evaluation of data quality across multiple reviewers, populations, and continuous\/categorical data.<\/jats:p>","DOI":"10.1177\/0962280214534651","type":"journal-article","created":{"date-parts":[[2014,5,15]],"date-time":"2014-05-15T22:04:34Z","timestamp":1400191474000},"page":"2939-2958","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":51,"title":["Choice of agreement indices for assessing and improving measurement reproducibility in a core laboratory setting"],"prefix":"10.1177","volume":"25","author":[{"given":"Huiman X","family":"Barnhart","sequence":"first","affiliation":[{"name":"Duke Clinical Research Institute, Duke University Medical Center, Durham, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Eric","family":"Yow","sequence":"additional","affiliation":[{"name":"Duke Clinical Research Institute, Duke University Medical Center, Durham, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Anna Lisa","family":"Crowley","sequence":"additional","affiliation":[{"name":"Duke Clinical Research Institute, Duke University Medical Center, Durham, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Melissa A","family":"Daubert","sequence":"additional","affiliation":[{"name":"Duke Clinical Research Institute, Duke University Medical Center, Durham, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dawn","family":"Rabineau","sequence":"additional","affiliation":[{"name":"Duke Clinical Research Institute, Duke University Medical Center, Durham, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Robert","family":"Bigelow","sequence":"additional","affiliation":[{"name":"Duke Clinical Research Institute, Duke University Medical Center, Durham, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michael","family":"Pencina","sequence":"additional","affiliation":[{"name":"Duke Clinical Research Institute, Duke University Medical Center, Durham, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pamela S","family":"Douglas","sequence":"additional","affiliation":[{"name":"Duke Clinical Research Institute, Duke University Medical Center, Durham, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"179","published-online":{"date-parts":[[2016,7,11]]},"reference":[{"key":"bibr1-0962280214534651","doi-asserted-by":"publisher","DOI":"10.1161\/CIRCULATIONAHA.107.743120"},{"key":"bibr2-0962280214534651","first-page":"1086","volume":"17","author":"Gottdiener JS","year":"2004","journal-title":"J Am Soc Echocardiogr"},{"key":"bibr3-0962280214534651","doi-asserted-by":"publisher","DOI":"10.1016\/j.echo.2009.05.020"},{"key":"bibr4-0962280214534651","doi-asserted-by":"publisher","DOI":"10.1080\/10543400701376480"},{"key":"bibr5-0962280214534651","first-page":"775","volume":"53","author":"Atkinson G","year":"1997","journal-title":"Biometrics"},{"key":"bibr6-0962280214534651","volume-title":"Measures of interobserver agreement and reliability","author":"Shoukri MM","year":"2011","edition":"2"},{"key":"bibr7-0962280214534651","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclinepi.2010.03.002"},{"key":"bibr8-0962280214534651","doi-asserted-by":"publisher","DOI":"10.1016\/j.csda.2008.09.026"},{"key":"bibr9-0962280214534651","doi-asserted-by":"publisher","DOI":"10.2307\/2529310"},{"key":"bibr10-0962280214534651","doi-asserted-by":"publisher","DOI":"10.2307\/2532051"},{"key":"bibr11-0962280214534651","doi-asserted-by":"publisher","DOI":"10.1111\/j.0006-341X.2002.01020.x"},{"key":"bibr12-0962280214534651","doi-asserted-by":"publisher","DOI":"10.1080\/10543400701329489"},{"key":"bibr13-0962280214534651","doi-asserted-by":"crossref","unstructured":"Haber M, Gao J and Barnhart HX. Assessing observer agreement in studies involving replicated binary observations.\n                      J Biopharm Stat\n                      2007;\n                      17\n                      : 757\u2013766.","DOI":"10.1080\/10543400701329547"},{"key":"bibr14-0962280214534651","doi-asserted-by":"publisher","DOI":"10.1177\/096228029900800204"},{"key":"bibr15-0962280214534651","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1097-0258(20000130)19:2<255::AID-SIM293>3.0.CO;2-8"},{"key":"bibr16-0962280214534651","doi-asserted-by":"publisher","DOI":"10.1198\/016214502753479392"},{"key":"bibr17-0962280214534651","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4614-0562-7"},{"key":"bibr18-0962280214534651","first-page":"A14418","volume":"126","author":"Daubert MA","year":"2012","journal-title":"Circulation"},{"key":"bibr19-0962280214534651","unstructured":"Barnhart HX. Assessing agreement with relative area under the coverage probability curve. In:\n                      Abstract and oral presentation at the 2013 Joint Statistical Meetings\n                      , Montreal, Canada, 3\u20138 August 2013."}],"container-title":["Statistical Methods in Medical Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/0962280214534651","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/0962280214534651","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/0962280214534651","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,1]],"date-time":"2026-05-01T23:47:12Z","timestamp":1777679232000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/0962280214534651"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,7,11]]},"references-count":19,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2016,12]]}},"alternative-id":["10.1177\/0962280214534651"],"URL":"https:\/\/doi.org\/10.1177\/0962280214534651","relation":{},"ISSN":["0962-2802","1477-0334"],"issn-type":[{"value":"0962-2802","type":"print"},{"value":"1477-0334","type":"electronic"}],"subject":[],"published":{"date-parts":[[2016,7,11]]}}}