{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,3]],"date-time":"2026-09-03T14:13:48Z","timestamp":1788444828526,"version":"build-2803163510"},"reference-count":55,"publisher":"BMJ","issue":"7","license":[{"start":{"date-parts":[[2026,7,22]],"date-time":"2026-07-22T00:00:00Z","timestamp":1784678400000},"content-version":"unspecified","delay-in-days":21,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Deutsche Forschungsgemeinschaft (DFG, German Research Foundation","award":["560535232"],"award-info":[{"award-number":["560535232"]}]}],"content-domain":{"domain":["bmj.com"],"crossmark-restriction":true},"short-container-title":["BMJ Open"],"accepted":{"date-parts":[[2026,7,2]]},"published-print":{"date-parts":[[2026,7]]},"abstract":"<jats:sec>\n                    <jats:title>Introduction<\/jats:title>\n                    <jats:p>The Guidelines for Reporting Reliability and Agreement Studies (GRRAS) were developed to improve the completeness and transparency of reporting of reliability and agreement of health measurement instruments. However, since their publication in 2011 methodological standards, both for reporting guideline development and reliability, agreement and measurement error studies have advanced, highlighting the need for aligning and updating. In addition, related initiatives like COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) have emerged, offering opportunities to harmonise terminology and promote consistency across diverse types of measurement instruments. This method review aims to (1) systematically identify and synthesise commentaries on and evaluations of the original GRRAS and (2) map recent methodological developments in the planning, conduct and interpretation of reliability, agreement and measurement error studies in health science, psychology and education that should be reflected in the reporting, to inform the development of the GRRAS-COSMIN reporting guidelines.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods and analysis<\/jats:title>\n                    <jats:p>Two complementary search strategies will be employed. First, forward direct citation tracking of the original GRRAS publications will be conducted in Web of Science. We will include sources providing critique, commentaries or suggestions related to the GRRAS. We will exclude publications that used GRRAS just for structuring their report, withdrawn or retracted articles, textbooks, peer reviews, supplements and conference materials without full-text publications. Included articles will be analysed using descriptive content analysis. Second, we will perform a systematic search in MEDLINE, PsycInfo, Embase, ERIC and CINAHL supplemented by key references identified by or known to the author team. We will include studies, reviews, commentaries, editorials, methodological papers, tutorials and guidance documents that discuss or critically reflect on methodological aspects of the measurement properties reliability and\/or agreement\/measurement error published between 1 January 2015 and 30 June 2026 in health sciences, psychology or education. Eligibility will be assessed by two independent reviewers and included sources will be analysed using \u2018codebook\u2019 thematic analysis. Findings will be presented narratively supplemented by visuals and tables if appropriate.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Ethics and dissemination<\/jats:title>\n                    <jats:p>This study involves publicly available data. No ethical approval is needed. Results will be disseminated through an open-access journal publication following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses-Scoping Review Extension and conference presentations.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1136\/bmjopen-2025-109876","type":"journal-article","created":{"date-parts":[[2026,7,27]],"date-time":"2026-07-27T21:04:21Z","timestamp":1785186261000},"page":"e109876","update-policy":"https:\/\/doi.org\/10.1136\/crossmarkpolicy","source":"Crossref","is-referenced-by-count":0,"title":["Developments in the field of reliability and agreement studies in health, psychological and educational measurement: a methods review protocol"],"prefix":"10.1136","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7487-7898","authenticated-orcid":false,"given":"M-M","family":"Pittelkow","sequence":"first","affiliation":[{"name":"Institute of Clinical Nursing Science, Charit\u00e9 \u2013 University Medical Center Berlin, corporate member of Freie Universit\u00e4t Berlin and Humboldt-Universit\u00e4t zu Berlin, Berlin, Germany"},{"name":"QUEST Center for Responsible Research, Berlin Institute of Health at Charit\u00e9 \u2013 Universit\u00e4tsmedizin Berlin, Berlin, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"M","family":"M\u00f6nch","sequence":"additional","affiliation":[{"name":"Institute of Biometry and Clinical Epidemiology, Charit\u00e9 \u2013 University Medical Center Berlin, corporate member of Freie Universit\u00e4t Berlin and Humboldt-Universit\u00e4t zu Berlin, Berlin, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"L B","family":"Mokkink","sequence":"additional","affiliation":[{"name":"Department of Epidemiology and Data Science, Amsterdam UMC location Vrije Universiteit Amsterdam, Amsterdam, The Netherlands"},{"name":"Methodology, Amsterdam Public Health Research Institute, Amsterdam, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0610-6271","authenticated-orcid":false,"given":"D L","family":"Streiner","sequence":"additional","affiliation":[{"name":"Department of Psychiatry and Behavioural Neurosciences, McMaster University, Hamilton, Ontario, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5337-758X","authenticated-orcid":false,"given":"Stig","family":"Brorson","sequence":"additional","affiliation":[{"name":"Centre for Evidence-Based Orthopaedics, Department of Orthopaedic Surgery, Zealand University Hospital, K\u00f8ge, Denmark"},{"name":"Department of Clinical Medicine, University of Copenhagen, Copenhagen, Denmark"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"M","family":"Shoukri","sequence":"additional","affiliation":[{"name":"Biostatistics and Scientific Computing Department, King Faisal Specialist Hospital and Research Centre, Riyadh, Saudi Arabia"},{"name":"Department of Epidemiology and Biostatistics, Schulich School of Medicine and Dentistry, University of Western Ontario, London, Ontario, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0750-3818","authenticated-orcid":false,"given":"J","family":"Kottner","sequence":"additional","affiliation":[{"name":"Institute of Clinical Nursing Science, Charit\u00e9 \u2013 University Medical Center Berlin, corporate member of Freie Universit\u00e4t Berlin and Humboldt-Universit\u00e4t zu Berlin, Berlin, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"239","published-online":{"date-parts":[[2026,7,22]]},"reference":[{"key":"2026090306253601000_16.7.e109876.1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclinepi.2010.03.002"},{"key":"2026090306253601000_16.7.e109876.2","doi-asserted-by":"crossref","first-page":"193","DOI":"10.2147\/PROM.S398886","article-title":"Studies on Reliability and Measurement Error of Measurements in Medicine - From Design to Statistics Explained for Medical Researchers","volume":"14","author":"Mokkink","year":"2023","journal-title":"Patient Relat Outcome Meas"},{"key":"2026090306253601000_16.7.e109876.3","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijnurstu.2011.01.016"},{"key":"2026090306253601000_16.7.e109876.4","unstructured":"What is a reporting guideline? | EQUATOR network. Available: https:\/\/www.equator-network.org\/about-us\/what-is-a-reporting-guideline\/ [Accessed 3 Jun 2025]."},{"key":"2026090306253601000_16.7.e109876.5","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclinepi.2010.12.001"},{"key":"2026090306253601000_16.7.e109876.6","doi-asserted-by":"crossref","first-page":"305","DOI":"10.1097\/JCP.0000000000000517","article-title":"Statistics Commentary Series: Commentary #15\u2014Reliability","volume":"36","author":"Streiner","year":"2016","journal-title":"J Clin Psychopharmacol"},{"key":"2026090306253601000_16.7.e109876.7","doi-asserted-by":"crossref","first-page":"926","DOI":"10.1016\/j.ijnurstu.2009.12.018","article-title":"Internal consistency and Cronbach\u2019s \u03b1: A comment on Beeckman et al. (2010)","volume":"47","author":"Kottner","year":"2010","journal-title":"Int J Nurs Stud"},{"key":"2026090306253601000_16.7.e109876.8","doi-asserted-by":"crossref","DOI":"10.1016\/j.mex.2023.102212","article-title":"Gwet\u2019s AC1 is not a substitute for Cohen\u2019s kappa - A comparison of basic properties","volume":"10","author":"Vach","year":"2023","journal-title":"MethodsX"},{"key":"2026090306253601000_16.7.e109876.9","doi-asserted-by":"crossref","first-page":"103694","DOI":"10.1016\/j.ijnurstu.2020.103694","article-title":"Measurement properties of classifications for skin tears: A systematic review","volume":"110","author":"Van Tiggelen","year":"2020","journal-title":"Int J Nurs Stud"},{"key":"2026090306253601000_16.7.e109876.10","doi-asserted-by":"publisher","DOI":"10.1016\/j.ridd.2010.09.001"},{"key":"2026090306253601000_16.7.e109876.11","doi-asserted-by":"crossref","DOI":"10.1136\/bmj.f2125","article-title":"Clinicians are right not to like Cohen\u2019s","volume":"346","author":"de Vet","year":"2013","journal-title":"BMJ"},{"key":"2026090306253601000_16.7.e109876.12","doi-asserted-by":"crossref","first-page":"5060","DOI":"10.1002\/sim.10217","article-title":"Asymptotic Confidence Interval, Sample Size Formulas and Comparison Test for the Agreement Intra-Class Correlation Coefficient in Inter-Rater Reliability Studies","volume":"43","author":"Bourredjem","year":"2024","journal-title":"Stat Med"},{"key":"2026090306253601000_16.7.e109876.13","doi-asserted-by":"crossref","unstructured":"Lu M-J , Zhong W-H , Liu Y-X , et al . Sample Size for Assessing Agreement between Two Methods of Measurement by Bland-Altman Method. Int J Biostat 2016;12. doi:10.1515\/ijb-2015-0039","DOI":"10.1515\/ijb-2015-0039"},{"key":"2026090306253601000_16.7.e109876.14","doi-asserted-by":"publisher","DOI":"10.1016\/j.annepidem.2014.06.097"},{"key":"2026090306253601000_16.7.e109876.15","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1007\/s10742-022-00293-9","article-title":"Sample size recommendations for studies on reliability and measurement error: an online application based on simulation studies","volume":"23","author":"Mokkink","year":"2023","journal-title":"Health Serv Outcomes Res Method"},{"key":"2026090306253601000_16.7.e109876.16","doi-asserted-by":"crossref","unstructured":"Oster C , Schoo A , Litt J , et al . Supporting workforce practice change: protocol for a pilot study of a motivational interviewing virtual client software tool for health professionals. BMJ Open 2020;10. doi:10.1136\/bmjopen-2019-033080","DOI":"10.1136\/bmjopen-2019-033080"},{"key":"2026090306253601000_16.7.e109876.17","doi-asserted-by":"crossref","first-page":"69","DOI":"10.3390\/diagnostics8040069","article-title":"Experience Applying the Guidelines for Reporting Reliability and Agreement Studies (GRRAS) Indicated Five Questions Should Be Addressed in the Planning Phase from a Statistical Point of View","volume":"8","author":"Gerke","year":"2018","journal-title":"Diagnostics"},{"key":"2026090306253601000_16.7.e109876.18","doi-asserted-by":"crossref","first-page":"1901","DOI":"10.1007\/s11136-025-03950-x","article-title":"COSMIN reporting guideline for studies on measurement properties of patient\u2011reported outcome measures: version 2.0","volume":"34","author":"Gagnier","year":"2025","journal-title":"Qual Life Res"},{"key":"2026090306253601000_16.7.e109876.19","doi-asserted-by":"crossref","unstructured":"Elsman EBM , Mokkink LB , Terwee CB , et al . Guideline for reporting systematic reviews of outcome measurement instruments (OMIs): PRISMA-COSMIN for OMIs 2024. J Patient-Rep Outcomes 2024;8. doi:10.1186\/s41687-024-00727-7","DOI":"10.1186\/s41687-024-00727-7"},{"key":"2026090306253601000_16.7.e109876.20","doi-asserted-by":"publisher","DOI":"10.1007\/s11136-024-03761-6"},{"key":"2026090306253601000_16.7.e109876.21","doi-asserted-by":"crossref","unstructured":"Mokkink LB , Boers M . COSMIN Risk of Bias tool to assess the quality of studies on reliability or measurement error of outcome measurement instruments: a Delphi study. BMC Med Res Methodol 2020. doi:10.1186\/s12874-020-01179-5","DOI":"10.21203\/rs.3.rs-40864\/v2"},{"key":"2026090306253601000_16.7.e109876.22","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclinepi.2009.10.002"},{"key":"2026090306253601000_16.7.e109876.23","doi-asserted-by":"crossref","DOI":"10.1136\/bmjopen-2016-012799","article-title":"STARD 2015 guidelines for reporting diagnostic accuracy studies: explanation and elaboration","volume":"6","author":"Cohen","year":"2016","journal-title":"BMJ Open"},{"key":"2026090306253601000_16.7.e109876.24","doi-asserted-by":"crossref","DOI":"10.1136\/bmj.h5527","article-title":"STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies","volume":"351","author":"Bossuyt","year":"2015","journal-title":"BMJ"},{"key":"2026090306253601000_16.7.e109876.25","doi-asserted-by":"publisher","DOI":"10.1059\/0003-4819-155-8-201110180-00009"},{"key":"2026090306253601000_16.7.e109876.26","doi-asserted-by":"publisher","DOI":"10.1186\/1471-2288-3-25"},{"key":"2026090306253601000_16.7.e109876.27","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pmed.1000217","article-title":"Guidance for Developers of Health Research Reporting Guidelines","volume":"7","author":"Moher","year":"2010","journal-title":"PLoS Med"},{"key":"2026090306253601000_16.7.e109876.28","doi-asserted-by":"crossref","first-page":"953","DOI":"10.11124\/JBIES-21-00242","article-title":"Best practice guidance and reporting items for the development of scoping review protocols","volume":"20","author":"Peters","year":"2022","journal-title":"JBI Evidence Synthesis"},{"key":"2026090306253601000_16.7.e109876.29","unstructured":"Clarke M . Guide to the contents of a cochrane methodology protocol and review. 2020. Available: https:\/\/www.cochrane.org\/sites\/default\/files\/uploads\/PDFs\/guide_to_the_contents_of_a_cochrane_methodology_protocol_and_review.pdf"},{"key":"2026090306253601000_16.7.e109876.30","doi-asserted-by":"crossref","first-page":"46","DOI":"10.1177\/1094428120943281","article-title":"Best-Practice Recommendations for Producers, Evaluators, and Users of Methodological Literature Reviews","volume":"26","author":"Aguinis","year":"2023","journal-title":"Organ Res Methods"},{"key":"2026090306253601000_16.7.e109876.31","doi-asserted-by":"crossref","first-page":"111201","DOI":"10.1016\/j.jclinepi.2023.10.017","article-title":"Searching a methods topic: practical challenges and implications for search design","volume":"166","author":"Hirt","year":"2024","journal-title":"J Clin Epidemiol"},{"key":"2026090306253601000_16.7.e109876.32","doi-asserted-by":"crossref","DOI":"10.1186\/s13643-016-0343-0","article-title":"Reviewing the research methods literature: principles and strategies illustrated by a systematic overview of sampling in qualitative research","volume":"5","author":"Gentles","year":"2016","journal-title":"Syst Rev"},{"key":"2026090306253601000_16.7.e109876.33","doi-asserted-by":"publisher","DOI":"10.7326\/M18-0850"},{"key":"2026090306253601000_16.7.e109876.34","unstructured":"DeepL privacy policy | protecting your privacy is important to us. Available: https:\/\/www.deepl.com\/en\/privacy [Accessed 1 Jul 2026]."},{"key":"2026090306253601000_16.7.e109876.35","unstructured":"KI-Plattform von Deepl: \u00dcbersetzung, Voice und API. Available: https:\/\/www.deepl.com [Accessed 1 Jun 2026]."},{"key":"2026090306253601000_16.7.e109876.36","doi-asserted-by":"crossref","DOI":"10.1136\/bmj.f2125","article-title":"Clinicians are right not to like Cohen\u2019s \u03ba","volume":"346","author":"de Vet","year":"2013","journal-title":"BMJ"},{"key":"2026090306253601000_16.7.e109876.37","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclinepi.2020.01.008"},{"key":"2026090306253601000_16.7.e109876.38","doi-asserted-by":"crossref","first-page":"206","DOI":"10.1186\/s13643-024-02619-9","article-title":"Automation of duplicate record detection for systematic reviews: Deduplicator","volume":"13","author":"Forbes","year":"2024","journal-title":"Syst Rev"},{"key":"2026090306253601000_16.7.e109876.39","unstructured":"Think. Check. Submit . Identify trusted publishers for your research. Available: https:\/\/thinkchecksubmit.org\/ [Accessed 5 Jun 2026]."},{"key":"2026090306253601000_16.7.e109876.40","unstructured":"Organisation for Economic Co-operation and Development . OECD schema to web of science category mapping. 2022. Available: https:\/\/incites.zendesk.com\/hc\/en-gb\/articles\/22516984338321-OECD-Category-Schema#h_01HPJA09BSM1J5AK92WXGD9FMC"},{"key":"2026090306253601000_16.7.e109876.41","unstructured":"VERBI Software . MAXQDA (Version 24)."},{"key":"2026090306253601000_16.7.e109876.42","doi-asserted-by":"publisher","DOI":"10.1177\/1049732305276687"},{"key":"2026090306253601000_16.7.e109876.43","doi-asserted-by":"publisher","DOI":"10.1016\/j.nedt.2003.10.001"},{"key":"2026090306253601000_16.7.e109876.44","doi-asserted-by":"crossref","first-page":"328","DOI":"10.1080\/14780887.2020.1769238","article-title":"One size fits all? What counts as quality practice in (reflexive) thematic analysis?","volume":"18","author":"Braun","year":"2021","journal-title":"Qual Res Psychol"},{"key":"2026090306253601000_16.7.e109876.45","doi-asserted-by":"crossref","unstructured":"De Vet HCW , Terwee CB , Mokkink LB , et al . Measurement in Medicine: A Practical Guide. 1st edn. Cambridge University Press, 2011.","DOI":"10.1017\/CBO9780511996214"},{"key":"2026090306253601000_16.7.e109876.46","doi-asserted-by":"publisher","DOI":"10.1111\/jan.12402"},{"key":"2026090306253601000_16.7.e109876.47","doi-asserted-by":"crossref","unstructured":"Streiner DL , Norman GR , Cairney J . Health Measurement Scales: A Practical Guide to Their Development and Use. 6th edn. Oxford, New York: Oxford University Press, 2024.","DOI":"10.1093\/med\/9780192869487.001.0001"},{"key":"2026090306253601000_16.7.e109876.48","doi-asserted-by":"crossref","DOI":"10.1186\/s13063-016-1555-2","article-title":"How to select outcome measurement instruments for outcomes included in a \u201cCore Outcome Set\u201d \u2013 a practical guideline","volume":"17","author":"Prinsen","year":"2016","journal-title":"Trials"},{"key":"2026090306253601000_16.7.e109876.49","doi-asserted-by":"crossref","first-page":"716","DOI":"10.1111\/iwj.13574","article-title":"Reliability and agreement of instrumental skin barrier measurements in clinical pressure ulcer prevention research","volume":"18","author":"Kottner","year":"2021","journal-title":"Int Wound J"},{"key":"2026090306253601000_16.7.e109876.50","doi-asserted-by":"crossref","unstructured":"Altman DG , Moher D . Importance of transparent reporting of health research. In: Guidelines for Reporting Health Research: a User\u2019s Manual. 2014: 1\u201313.","DOI":"10.1002\/9781118715598.ch1"},{"key":"2026090306253601000_16.7.e109876.51","doi-asserted-by":"crossref","unstructured":"Simera I , Altman DG . Writing a research article that is \u201cfit for purpose\u201d: EQUATOR Network and reporting guidelines. Ann Intern Med 2009;151. doi:10.7326\/0003-4819-151-4-200908180-02002","DOI":"10.7326\/0003-4819-151-4-200908180-02002"},{"key":"2026090306253601000_16.7.e109876.52","doi-asserted-by":"crossref","unstructured":"Altman DG , Simera I . Using reporting guidelines effectively to ensure good reporting of health research. In: Guidelines for Reporting Health Research: A User\u2019s Manual. Wiley, 2014: 32\u201340.","DOI":"10.1002\/9781118715598.ch4"},{"key":"2026090306253601000_16.7.e109876.53","doi-asserted-by":"crossref","unstructured":"Field SM , Pownall M . Subjectivity is a feature, not a flaw: a call to unsilence the human element in science. Open Science Framework [Preprint] 2025. doi:10.31219\/osf.io\/ga5fb_v1","DOI":"10.31219\/osf.io\/ga5fb_v1"},{"key":"2026090306253601000_16.7.e109876.54","unstructured":"COSMIN definitions of domains, measurement properties, and aspects of measurement properties. Available: https:\/\/cosmin.nl\/wp-content\/uploads\/COSMIN-definitions-domains-measurement-properties.pdf [Accessed 19 Aug 2025]."},{"key":"2026090306253601000_16.7.e109876.55","unstructured":"American Educational Research Association, American Psychological Association, National Council on Measurement in Education . Standards for Educational and Psychological Testing. Washington, DC: American Educational Research Association, 2014."}],"container-title":["BMJ Open"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/syndication.highwire.org\/content\/doi\/10.1136\/bmjopen-2025-109876","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,3]],"date-time":"2026-09-03T13:26:12Z","timestamp":1788441972000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmjopen.bmj.com\/lookup\/doi\/10.1136\/bmjopen-2025-109876"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,7]]},"references-count":55,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2026,7,22]]},"published-print":{"date-parts":[[2026,7]]}},"alternative-id":["10.1136\/bmjopen-2025-109876"],"URL":"https:\/\/doi.org\/10.1136\/bmjopen-2025-109876","relation":{},"ISSN":["2044-6055","2044-6055"],"issn-type":[{"value":"2044-6055","type":"print"},{"value":"2044-6055","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,7]]}}}