{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,5]],"date-time":"2026-01-05T22:10:01Z","timestamp":1767651001398},"reference-count":34,"publisher":"Wiley","issue":"23-24","license":[{"start":{"date-parts":[[2006,10,15]],"date-time":"2006-10-15T00:00:00Z","timestamp":1160870400000},"content-version":"vor","delay-in-days":4322,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#vor"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Statistics in Medicine"],"published-print":{"date-parts":[[1994,12,15]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>We introduce a new statistical method, which separates and measures different types of variability between paired ordered categorical measurements. The key to the separation is a two\u2010way augmented ranking approach of observations in a contingency table. It means that cases classified in a specific category by one rater will be internally ranked according to the classifications from the other. This enables us to extract the component of interobserver variation which is not systematic. The variance of the rank differences between judgements is a suitable measure of this interrater variability, which we characterize as random. The empirical measure of random interjudge disagreement, which lies between 0 and 1, is called the relative rank variance and is an estimate of a parameter defined on the multinomial probability distribution in the contingency table. The systematic differences are determined by the marginals and described by two empirical measures, relative position and relative concentration; both measures lie between \u2010 1 and 1. Our method is applied to data sets from a reliability study of two clinical rating scales for assessing hydrocephalus and subarachnoid haemorrhage.<\/jats:p>","DOI":"10.1002\/sim.4780132308","type":"journal-article","created":{"date-parts":[[2007,4,28]],"date-time":"2007-04-28T18:17:46Z","timestamp":1177784266000},"page":"2437-2453","source":"Crossref","is-referenced-by-count":47,"title":["Separation of systematic and random differences in ordinal rating scales"],"prefix":"10.1002","volume":"13","author":[{"given":"Elisabeth","family":"Svensson","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sture","family":"Holm","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"311","published-online":{"date-parts":[[2006,10,15]]},"reference":[{"key":"e_1_2_1_2_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.103.2684.677"},{"key":"e_1_2_1_3_2","unstructured":"Svensson E.\u2018Analysis of systematic and random differences between paired ordinal categorical data\u2019 thesis Almqvist & Wiksell International Stockholm 1993."},{"key":"e_1_2_1_4_2","first-page":"1","volume-title":"The Design and Analysis of Clinical Experiments","author":"Fleiss J. L.","year":"1986"},{"key":"e_1_2_1_5_2","doi-asserted-by":"publisher","DOI":"10.1007\/BF02296208"},{"key":"e_1_2_1_6_2","doi-asserted-by":"publisher","DOI":"10.1177\/096228029200100205"},{"key":"e_1_2_1_7_2","unstructured":"Starmark J. E.\u2018Analysing \u201ccoma scales\u201d. The introduction of a new scale \u2010 the Reaction Level Scale (RLS85)\u2019 thesis University of G\u00f6teborg G\u00f6teborg 1988."},{"key":"e_1_2_1_8_2","doi-asserted-by":"publisher","DOI":"10.1038\/clpt.1981.18"},{"key":"e_1_2_1_9_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0303-8467(78)80067-5"},{"key":"e_1_2_1_10_2","doi-asserted-by":"publisher","DOI":"10.1136\/jnnp.42.12.1163"},{"key":"e_1_2_1_11_2","doi-asserted-by":"publisher","DOI":"10.2307\/2529310"},{"key":"e_1_2_1_12_2","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.304.6840.1491"},{"key":"e_1_2_1_13_2","first-page":"758","article-title":"A methodological framework for developing and selecting endpoints in clinical trials","volume":"9","author":"Tugwell P.","year":"1982","journal-title":"Journal of Rheumatology"},{"key":"e_1_2_1_14_2","doi-asserted-by":"publisher","DOI":"10.1016\/0165-0327(90)90055-D"},{"key":"e_1_2_1_15_2","doi-asserted-by":"publisher","DOI":"10.1056\/NEJM197509252931307"},{"key":"e_1_2_1_16_2","doi-asserted-by":"publisher","DOI":"10.1192\/bjp.125.3.248"},{"key":"e_1_2_1_17_2","doi-asserted-by":"publisher","DOI":"10.3171\/jns.1983.58.3.0321"},{"key":"e_1_2_1_18_2","doi-asserted-by":"publisher","DOI":"10.1016\/0140-6736(91)91309-I"},{"key":"e_1_2_1_19_2","unstructured":"Svensson E. von Essen C. Ekholm S. Johansson A. Holm S.andStarmark J. E. \u2018Interobserver variability in the assessment of subarachnoid blood and acute hydrocephalus on CT scans\u2019 1993(to be published)"},{"key":"e_1_2_1_20_2","doi-asserted-by":"publisher","DOI":"10.1227\/00006123-198001000-00001"},{"key":"e_1_2_1_21_2","doi-asserted-by":"publisher","DOI":"10.2307\/2531866"},{"key":"e_1_2_1_22_2","doi-asserted-by":"publisher","DOI":"10.1097\/00004424-197903000-00002"},{"key":"e_1_2_1_23_2","doi-asserted-by":"publisher","DOI":"10.1037\/0033-2909.99.1.100"},{"key":"e_1_2_1_24_2","doi-asserted-by":"publisher","DOI":"10.1177\/0272989X8800800309"},{"key":"e_1_2_1_25_2","doi-asserted-by":"publisher","DOI":"10.2307\/2527727"},{"key":"e_1_2_1_26_2","first-page":"1","article-title":"The jack\u2010knife \u2010 a review","volume":"61","author":"Miller R. G.","year":"1974","journal-title":"Biometrics"},{"key":"e_1_2_1_27_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4899-4541-9"},{"key":"e_1_2_1_28_2","doi-asserted-by":"publisher","DOI":"10.1177\/001316446002000104"},{"key":"e_1_2_1_29_2","doi-asserted-by":"publisher","DOI":"10.1037\/0033-2909.104.3.405"},{"key":"e_1_2_1_30_2","unstructured":"Schouten H. J. A.\u2018Statistical measurement of interobserver agreement\u2019 thesis Erasmus Universitet Rotterdam 1985."},{"key":"e_1_2_1_31_2","doi-asserted-by":"publisher","DOI":"10.1093\/aje\/126.2.161"},{"key":"e_1_2_1_32_2","doi-asserted-by":"publisher","DOI":"10.1177\/001316448904900407"},{"key":"e_1_2_1_33_2","doi-asserted-by":"publisher","DOI":"10.1016\/0895-4356(90)90158-L"},{"key":"e_1_2_1_34_2","doi-asserted-by":"publisher","DOI":"10.1016\/0895-4356(88)90031-5"},{"key":"e_1_2_1_35_2","first-page":"17","volume-title":"Central Nervous System Trauma Status Report","author":"Miller J. D.","year":"1985"}],"container-title":["Statistics in Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/api.wiley.com\/onlinelibrary\/tdm\/v1\/articles\/10.1002%2Fsim.4780132308","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/sim.4780132308","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,8,31]],"date-time":"2023-08-31T08:16:45Z","timestamp":1693469805000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/sim.4780132308"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[1994,12,15]]},"references-count":34,"journal-issue":{"issue":"23-24","published-print":{"date-parts":[[1994,12,15]]}},"alternative-id":["10.1002\/sim.4780132308"],"URL":"https:\/\/doi.org\/10.1002\/sim.4780132308","archive":["Portico"],"relation":{},"ISSN":["0277-6715","1097-0258"],"issn-type":[{"value":"0277-6715","type":"print"},{"value":"1097-0258","type":"electronic"}],"subject":[],"published":{"date-parts":[[1994,12,15]]}}}