{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,2]],"date-time":"2026-06-02T18:04:26Z","timestamp":1780423466277,"version":"3.54.1"},"reference-count":27,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2019,1,7]],"date-time":"2019-01-07T00:00:00Z","timestamp":1546819200000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Med Res Methodol"],"published-print":{"date-parts":[[2019,12]]},"DOI":"10.1186\/s12874-018-0651-2","type":"journal-article","created":{"date-parts":[[2019,1,8]],"date-time":"2019-01-08T09:11:06Z","timestamp":1546938666000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":14,"title":["Providing quality data in health care - almost perfect inter-rater agreement in the Norwegian tonsil surgery register"],"prefix":"10.1186","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1634-584X","authenticated-orcid":false,"given":"Siri","family":"Wennberg","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lasse A.","family":"Karlsen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Joacim","family":"Stalfors","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mette","family":"Bratt","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Vegard","family":"Bugten","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2019,1,7]]},"reference":[{"key":"651_CR1","doi-asserted-by":"crossref","unstructured":"Larsson S, Lawyer P, Garellick G, Lindahl B, Lundstr\u00f6m M. Use Of 13 Disease Registries In 5 Countries Demonstrates The potential to use outcome data to improve health care\u2019s value. Health Affairs 5 31, NO 1 2012:220\u2013227.","DOI":"10.1377\/hlthaff.2011.0762"},{"key":"651_CR2","doi-asserted-by":"crossref","unstructured":"McNeil J, Evans S, Johnson N, Cameron P. Clinical-quality registries: their role in quality improvement. MJA. 2010;192(5).","DOI":"10.5694\/j.1326-5377.2010.tb03499.x"},{"key":"651_CR3","unstructured":"EyeNet Sweden. Handbook for establishing quality registries. 2005. \n                    http:\/\/demo.web4u.nu\/eyenet\/uploads\/Handboken%20engelsk%20version%20060306.pdf\n                    \n                   Accessed 2 Nov 2018."},{"key":"651_CR4","doi-asserted-by":"publisher","first-page":"2477","DOI":"10.1056\/NEJMp1011024","volume":"363","author":"ME Porter","year":"2010","unstructured":"Porter ME. What is value in health care? N Engl J Med. 2010;363:2477\u201381.","journal-title":"N Engl J Med"},{"key":"651_CR5","unstructured":"Norwegian Tonsil Surgery Register. Norsk Kvalitetsregister \u00d8re-Nese-Hals \u2013 Tonsilleregisteret. \u00c5rsrapport 2017. 2018. \n                    https:\/\/stolav.no\/seksjon\/norsk-tonsilleregister\/Documents\/Norsk_tonsilleregister_\u00e5rsrapport_2017.pdf\n                    \n                   Accessed 2 Nov 2018."},{"key":"651_CR6","unstructured":"Helseatlas, SKDE. Barnehelseatlas for Norge. En oversikt og analyse av forbruket av somatiske helsetjenester for barn 0\u201316 \u00e5r for \u00e5rene 2011\u20132014. 2015. \n                    https:\/\/helseatlas.no\/sites\/default\/files\/rapport_digitalt.pdf\n                    \n                   Accessed 2 Nov 2018."},{"key":"651_CR7","doi-asserted-by":"publisher","first-page":"1353","DOI":"10.1007\/s00405-018-4945-0","volume":"275","author":"J Ruohoalho","year":"2018","unstructured":"Ruohoalho J, \u00d8stvoll E, Bratt M, Bugten V, B\u00e4ck L, M\u00e4kitie A, Ovesen T, Stalfors J. Systematic review of tonsil surgery quality registers and introduction of the Nordic tonsil surgery register. European Archives of Oto-Rhino-Laryngology. 2018;275:1353\u201363.","journal-title":"European Archives of Oto-Rhino-Laryngology"},{"issue":"10","key":"651_CR8","doi-asserted-by":"publisher","first-page":"1096","DOI":"10.1080\/00016489.2017.1327122","volume":"137","author":"N Hallenst\u00e5l","year":"2017","unstructured":"Hallenst\u00e5l N, Sunnergren O, Ericsson E, Hemlin C, S\u00f6derman A-CH, Nerfeldt P, Odhagen E, Ryding M, Stalfors J. Tonsil surgery in Sweden 2013\u20132015. Indications, surgical methods and patient-reported outcomes from the National Tonsil Surgery Register. Acta Otolaryngol. 2017;137(10):1096\u2013103.","journal-title":"Acta Otolaryngol"},{"issue":"3","key":"651_CR9","doi-asserted-by":"publisher","first-page":"248","DOI":"10.1111\/coa.12361","volume":"40","author":"AC S\u00f6derman","year":"2015","unstructured":"S\u00f6derman AC, Odhagen E, Ericsson E, Hemlin C, Hultcrantz E, Sunnergren O, Stalfors J. Post-tonsillectomy haemorrhage rates are related to technique for dissection and for haemostasis. An analysis of 15734 patients in the National Tonsil Surgery Register in Sweden. Clin Otolaryngol. 2015;40(3):248\u201354.","journal-title":"Clin Otolaryngol"},{"key":"651_CR10","doi-asserted-by":"publisher","first-page":"1631","DOI":"10.1007\/s00405-018-4942-3","volume":"275","author":"E Odhagen","year":"2018","unstructured":"Odhagen E, Sunnergren O, S\u00f6derman AH, Thor J, Stalfors J. Reducing post-tonsillectomy haemorrhage rates through a quality improvement project using a Swedish national quality register: a case study. Eur Arch Otorhinolaryngol. 2018;275:1631\u20139.","journal-title":"Eur Arch Otorhinolaryngol"},{"issue":"11","key":"651_CR11","doi-asserted-by":"publisher","first-page":"2322","DOI":"10.1002\/lary.22179","volume":"121","author":"AC S\u00f6derman","year":"2011","unstructured":"S\u00f6derman AC, Ericsson E, Hemlin C, Hultcrantz E, Mansson I, Roos K, Stalfors J. Reduced risk of primary postoperative hemorrhage after tonsil surgery in Sweden: results from the national tonsil surgery register in Sweden covering more than 10 years and 54,696 operations. Laryngoscope. 2011;121(11):2322\u20136.","journal-title":"Laryngoscope"},{"issue":"2","key":"651_CR12","first-page":"142","volume":"106","author":"DJ Solomon","year":"1991","unstructured":"Solomon DJ, Henry RC, Hogan JG, Van Amburg GH, Taylor J. Evaluation and implementation of public health registries. Public Health Rep. 1991;106(2):142\u201350.","journal-title":"Public Health Rep"},{"key":"651_CR13","doi-asserted-by":"publisher","first-page":"584","DOI":"10.1186\/s13104-015-1556-3","volume":"8","author":"T Varmdal","year":"2015","unstructured":"Varmdal T, Ellekj\u00e6r H, Fj\u00e6rtoft H, Indredavik B, Lydersen S, B\u00f8naa K. Inter-rater reliability of a national acute stroke register. BMC Res Notes. 2015;8:584.","journal-title":"BMC Res Notes"},{"key":"651_CR14","doi-asserted-by":"publisher","first-page":"305","DOI":"10.2147\/CLEP.S105933","volume":"8","author":"RE Govatsmark","year":"2016","unstructured":"Govatsmark RE, Sneeggen S, Karlsaune H, Sl\u00f8rdahl SA, B\u00f8naa K. Interrater reliability of a national acute myocardial infarction register. Clin Epidemiol. 2016;8:305\u201312.","journal-title":"Clin Epidemiol"},{"key":"651_CR15","volume-title":"Handbook of inter-rater reliability. 4th ed","author":"KL Gwet","year":"2014","unstructured":"Gwet KL. Handbook of inter-rater reliability. 4th ed. Gaithersburg: Advanced Analytics LLC; 2014."},{"key":"651_CR16","unstructured":"User Manual for the Norwegian Tonsil Surgery Register. St. Olav\u2019s university hospital in Trondheim. 2017. \n                    https:\/\/stolav.no\/seksjon\/norsk-tonsilleregister\/Documents\/Brukermanual%20Tonsilleregisteret%20versjon%201.0.pdf\n                    \n                   Accessed 5 Nov 2018."},{"issue":"11","key":"651_CR17","doi-asserted-by":"publisher","first-page":"1511","DOI":"10.1002\/sim.4780111109","volume":"11","author":"A Donner","year":"1992","unstructured":"Donner A, Eliasziw M. A goodness-of-fit approach to inference procedures for the kappa statistic: confidence interval construction, significance-testing and sample size estimation. Stat Med. 1992;11(11):1511\u20139.","journal-title":"Stat Med"},{"issue":"Pt 1","key":"651_CR18","doi-asserted-by":"publisher","first-page":"29","DOI":"10.1348\/000711006X126600","volume":"61","author":"KL Gwet","year":"2008","unstructured":"Gwet KL. Computing inter-rater reliability and its variance in the presence of high agreement. Br J Math Stat Psychol. 2008;61(Pt 1):29\u201348.","journal-title":"Br J Math Stat Psychol"},{"issue":"1","key":"651_CR19","first-page":"37","volume":"20","author":"J Cohen","year":"1960","unstructured":"Cohen J. A coefficient of agreement for nominal scales. Educ PsycholMeas. 1960;20(1):37\u201346.","journal-title":"Educ PsycholMeas"},{"issue":"6","key":"651_CR20","doi-asserted-by":"publisher","first-page":"543","DOI":"10.1016\/0895-4356(90)90158-L","volume":"43","author":"AR Feinstein","year":"1990","unstructured":"Feinstein AR, Cicchetti DV. High agreement but low kappa: I. The problems of two paradoxes. J Clin Epidemiol. 1990;43(6):543\u20139.","journal-title":"J Clin Epidemiol"},{"issue":"5","key":"651_CR21","doi-asserted-by":"publisher","first-page":"423","DOI":"10.1016\/0895-4356(93)90018-V","volume":"46","author":"T Byrt","year":"1993","unstructured":"Byrt T, Bishop J, Carlin JB. Bias, prevalence and kappa. J Clin Epidemiol. 1993;46(5):423\u20139.","journal-title":"J Clin Epidemiol"},{"issue":"1","key":"651_CR22","doi-asserted-by":"publisher","first-page":"159","DOI":"10.2307\/2529310","volume":"33","author":"JR Landis","year":"1977","unstructured":"Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159\u201374.","journal-title":"Biometrics"},{"key":"651_CR23","doi-asserted-by":"crossref","unstructured":"Nahathai W, Wongpakaran T, Wedding D, Gwet KL. A comparison of cohen\u2019s kappa and Gwet\u2019s AC1 when calculating inter-rater reliability coefficients: a study conducted with personality disorder samples. BMC Med Res Methodol. 2013;13(61).","DOI":"10.1186\/1471-2288-13-61"},{"key":"651_CR24","doi-asserted-by":"publisher","first-page":"330","DOI":"10.1016\/j.sapharm.2012.04.004","volume":"9","author":"N Gisev","year":"2013","unstructured":"Gisev N, Pharm B, Bell JS, Chen TF. Interrater agreement and interrater reliability: key concepts, approaches, and applications. Research in Social and Administrative Pharmacy. 2013;9:330\u20138.","journal-title":"Research in Social and Administrative Pharmacy"},{"key":"651_CR25","unstructured":"Norwegian Tonsil Surgery Register. Aldersfordeling blant pasienter i \u00d8NH \u2013 Tonsilleregisteret i 2017. 2018. \n                    https:\/\/stolav.no\/Documents\/Aldersfordeling%20blant%20pasienter%20i%20%c3%98NH.pdf\n                    \n                   Accessed 28 June 2018."},{"key":"651_CR26","unstructured":"Nasjonalt servicemilj\u00f8 for medisinske kvalitetsregistre (SKDE). Valideringsh\u00e5ndboken \n                    https:\/\/www.kvalitetsregistre.no\/validering\n                    \n                   Accessed 14 Nov 2018."},{"key":"651_CR27","unstructured":"Norwegian Tonsil Surgery Register. Oversikt over operasjonsteknikk ved tonsillektomi og tonsillotomi i Norge i 2017. 2018. \n                    https:\/\/stolav.no\/Documents\/Oversikt%20over%20operasjonsteknikk%20ved%20tonsilleoperasjoner%20i%20Norge%20i%202017.pdf\n                    \n                   Accessed 28 June 2018."}],"container-title":["BMC Medical Research Methodology"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/s12874-018-0651-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1186\/s12874-018-0651-2\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/s12874-018-0651-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2020,1,6]],"date-time":"2020-01-06T19:11:43Z","timestamp":1578337903000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcmedresmethodol.biomedcentral.com\/articles\/10.1186\/s12874-018-0651-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,1,7]]},"references-count":27,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2019,12]]}},"alternative-id":["651"],"URL":"https:\/\/doi.org\/10.1186\/s12874-018-0651-2","relation":{},"ISSN":["1471-2288"],"issn-type":[{"value":"1471-2288","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,1,7]]},"assertion":[{"value":"4 September 2018","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"20 December 2018","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"7 January 2019","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and\/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. The study was submitted to the Regional Committee for Medical and Health Research Ethics (REC) as a remit assessment since we were in doubt as to whether our study had to be approved by the REC. The committee concluded that this was a quality improvement study validating register data against source data. The project was in accordance with The Norwegian Health Research Act \u00a7 2 and \u00a7 4 and was not required for submission and could therefore be implemented and published without the approval of the REC. Written informed consent was obtained from all individual participants included in the study, and on behalf of the minors in this study (under the age of 16) parents have signed a written informed consent. Patients who were minors at the time of inclusion in the register are contacted upon turning 16 and given the option of withdrawing the consent given by their parents, and having the information concerning themselves deleted from the register.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The inter-rater reliability study was performed by an employee of the register. We were aware of this when we designed the study. Therefore, the investigator was blinded to the registrations in the registry during the period the patient records were reviewed.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}},{"value":"Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Publisher\u2019s Note"}}],"article-number":"6"}}