{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,7]],"date-time":"2026-11-07T07:01:09Z","timestamp":1794034869416,"version":"4.8.0"},"reference-count":45,"publisher":"National Athletic Trainers' Association","issue":"3","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2014,6,1]]},"abstract":"<jats:sec><jats:title>Context:<\/jats:title><jats:p>Health care professions have replaced traditional multiple choice tests or essays with structured and practical, performance-based examinations with the hope of eliminating rater bias and measuring clinical competence.<\/jats:p><\/jats:sec><jats:sec><jats:title>Objective:<\/jats:title><jats:p>To establish the validity and reliability of the Standardized Orthopedic Assessment Tool (SOAT) as a measure of clinical competence of orthopaedic injury evaluation.<\/jats:p><\/jats:sec><jats:sec><jats:title>Design:<\/jats:title><jats:p>Descriptive laboratory study.<\/jats:p><\/jats:sec><jats:sec><jats:title>Setting:<\/jats:title><jats:p>University.<\/jats:p><\/jats:sec><jats:sec><jats:title>Patients or Other Participants:<\/jats:title><jats:p>A total of 60 undergraduate students and 11 raters from 3 Canadian universities and 1 standardized patient.<\/jats:p><\/jats:sec><jats:sec><jats:title>Intervention(s):<\/jats:title><jats:p>Students were required to complete a 30-minute musculoskeletal evaluation in 1 of 2 randomly assigned mock scenarios involving the knee (second-degree medial collateral ligament sprain) or the shoulder (third-degree supraspinatus muscle strain).<\/jats:p><\/jats:sec><jats:sec><jats:title>Main Outcome Measure(s):<\/jats:title><jats:p>We measured interreliability with an intraclass correlation coefficient (ICC) (2,k) and stability of the tool with standard error of measurement and confidence intervals. Agreement was measured using Bland-Altman plots. Concurrent validity was measured using a Pearson product moment correlation coefficient whereby the raters' global rating of a student was matched to the cumulative mean grade score.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results:<\/jats:title><jats:p>The ICCs were 0.75 and 0.82 for the shoulder and knee cases, respectively. Bland-Altman plots indicated no systematic bias between raters. In addition, Pearson product moment correlation analysis demonstrated a strong relationship between the overall cumulative mean grade score and the global rating score of the examinees' performances.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusions:<\/jats:title><jats:p>This study demonstrated good interrater reliability of the SOAT with a standard error of measurement that indicated very modest stability, strong agreement between raters, and correlation indicative of concurrent validity.<\/jats:p><\/jats:sec>","DOI":"10.4085\/1062-6050-49.1.12","type":"journal-article","created":{"date-parts":[[2014,2,17]],"date-time":"2014-02-17T18:27:37Z","timestamp":1392661657000},"page":"373-380","source":"Crossref","is-referenced-by-count":6,"title":["Validity and Reliability of the Standardized Orthopedic Assessment Tool (SOAT): A Variation of the Traditional Objective Structured Clinical Examination"],"prefix":"10.4085","volume":"49","author":[{"given":"Mark R.","family":"Lafave","sequence":"first","affiliation":[{"name":"Department of Physical Education and Recreation Studies, Mount Royal University, Calgary, AB, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Larry","family":"Katz","sequence":"first","affiliation":[{"name":"Faculty of Kinesiology, University of Calgary, AB, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"2557","reference":[{"issue":"2","key":"2020060411480919000_i1062-6050-49-3-373-b01","doi-asserted-by":"crossref","first-page":"161","DOI":"10.1002\/acr.20064","article-title":"Health care utilization for musculoskeletal disorders","volume":"62","author":"MacKay","year":"2010","journal-title":"Arthritis Care Res (Hoboken)"},{"issue":"1","key":"2020060411480919000_i1062-6050-49-3-373-b02","doi-asserted-by":"crossref","first-page":"26","DOI":"10.1377\/hlthaff.28.1.26","article-title":"Use of medical care for chronic conditions","volume":"28","author":"Decker","year":"2009","journal-title":"Health Aff (Millwood)"},{"issue":"534","key":"2020060411480919000_i1062-6050-49-3-373-b03","first-page":"7","article-title":"Measuring disease prevalence: a comparison of musculoskeletal disease using four general practice consultation databases","volume":"57","author":"Jordan","year":"2007","journal-title":"Br J Gen Pract"},{"issue":"12","key":"2020060411480919000_i1062-6050-49-3-373-b04","doi-asserted-by":"crossref","first-page":"2419","DOI":"10.3899\/jrheum.080308","article-title":"The current state of musculoskeletal clinical skills teaching for preclerkship medical students","volume":"35","author":"Oswald","year":"2008","journal-title":"J Rheumatol"},{"issue":"10","key":"2020060411480919000_i1062-6050-49-3-373-b05","doi-asserted-by":"crossref","first-page":"1121","DOI":"10.1007\/s10067-010-1527-y","article-title":"The inadequacies of musculoskeletal education","volume":"29","author":"Abou-Raya","year":"2010","journal-title":"Clin Rheumatol"},{"issue":"8","key":"2020060411480919000_i1062-6050-49-3-373-b06","first-page":"574","article-title":"Objective structured clinical exams: a critical review","volume":"40","author":"Turner","year":"2008","journal-title":"Fam Med"},{"issue":"5","key":"2020060411480919000_i1062-6050-49-3-373-b07","doi-asserted-by":"crossref","first-page":"447","DOI":"10.1097\/00001888-199605000-00012","article-title":"The emperor's new clothes: the OSCE reassessed","volume":"71","author":"Mavis","year":"1996","journal-title":"Acad Med"},{"issue":"3","key":"2020060411480919000_i1062-6050-49-3-373-b08","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1023\/A:1018369124027","article-title":"The risk of thoroughness: reliability and validity of global ratings and checklists in an OSCE","volume":"1","author":"Cunnington","year":"1996","journal-title":"Adv Health Sci Educ Theory Pract"},{"issue":"2","key":"2020060411480919000_i1062-6050-49-3-373-b09","doi-asserted-by":"crossref","first-page":"131","DOI":"10.1023\/A:1009878124073","article-title":"Global ratings of student performance in a standardized patient examination: is the whole more than the sum of the parts?","volume":"5","author":"Solomon","year":"2000","journal-title":"Adv Health Sci Educ Theory Pract"},{"issue":"7","key":"2020060411480919000_i1062-6050-49-3-373-b10","doi-asserted-by":"crossref","first-page":"725","DOI":"10.1097\/00001888-200207000-00018","article-title":"The effect of candidates' perceptions of the evaluation method on reliability of checklist and global rating scores in an objective structured clinical examination","volume":"77","author":"McIlroy","year":"2002","journal-title":"Acad Med"},{"issue":"9","key":"2020060411480919000_i1062-6050-49-3-373-b11","doi-asserted-by":"crossref","first-page":"993","DOI":"10.1097\/00001888-199809000-00020","article-title":"Comparing the psychometric properties of checklist and global rating scales for assessing performance on an OSCE-format examination","volume":"73","author":"Regehr","year":"1998","journal-title":"Acad Med"},{"issue":"1","key":"2020060411480919000_i1062-6050-49-3-373-b12","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s10459-005-4723-9","article-title":"Editorial: checklists vs. ratings, the illusion of objectivity, the demise of skills and the debasement of evidence","volume":"10","author":"Norman","year":"2005","journal-title":"Adv Health Sci Educ Theory Pract"},{"issue":"2","key":"2020060411480919000_i1062-6050-49-3-373-b13","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1111\/j.1365-2923.1991.tb00037.x","article-title":"Pitfalls in the pursuit of objectivity: issues of validity, efficiency and acceptability","volume":"25","author":"Norman","year":"1991","journal-title":"Med Educ"},{"issue":"10","key":"2020060411480919000_i1062-6050-49-3-373-b14","doi-asserted-by":"crossref","first-page":"1129","DOI":"10.1097\/00001888-199910000-00017","article-title":"OSCE checklists do not capture increasing levels of expertise","volume":"74","author":"Hodges","year":"1999","journal-title":"Acad Med"},{"issue":"10","key":"2020060411480919000_i1062-6050-49-3-373-b15","doi-asserted-by":"crossref","first-page":"901","DOI":"10.1046\/j.1365-2923.2002.01316.x","article-title":"The relationship between competence and performance: implications for assessing practice performance","volume":"36","author":"Rethans","year":"2002","journal-title":"Med Educ"},{"issue":"2","key":"2020060411480919000_i1062-6050-49-3-373-b16","doi-asserted-by":"crossref","first-page":"199","DOI":"10.1111\/j.1365-2923.2004.01755.x","article-title":"Techniques for measuring clinical competence: objective structured clinical examinations","volume":"38","author":"Newble","year":"2004","journal-title":"Med Educ"},{"issue":"9260","key":"2020060411480919000_i1062-6050-49-3-373-b17","doi-asserted-by":"crossref","first-page":"945","DOI":"10.1016\/S0140-6736(00)04221-5","article-title":"Assessment of clinical competence","volume":"357","author":"Wass","year":"2001","journal-title":"Lancet"},{"issue":"3","key":"2020060411480919000_i1062-6050-49-3-373-b18","doi-asserted-by":"crossref","first-page":"226","DOI":"10.1016\/S0002-9610(97)89597-9","article-title":"Testing technical skill via an innovative \u201cbench station\u201d examination","volume":"173","author":"Reznick","year":"1997","journal-title":"Am J Surg"},{"issue":"suppl 9","key":"2020060411480919000_i1062-6050-49-3-373-b19","doi-asserted-by":"crossref","first-page":"S63","DOI":"10.1097\/00001888-199009000-00045","article-title":"The assessment of clinical skills\/competence\/performance","volume":"65","author":"Miller","year":"1990","journal-title":"Acad Med"},{"issue":"2","key":"2020060411480919000_i1062-6050-49-3-373-b20","doi-asserted-by":"crossref","first-page":"180","DOI":"10.1016\/j.jsurg.2011.07.015","article-title":"Implementation of an objective structured clinical exam (OSCE) into orthopedic surgery residency training","volume":"69","author":"Griesser","year":"2012","journal-title":"J Surg Educ"},{"issue":"2","key":"2020060411480919000_i1062-6050-49-3-373-b21","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1097\/00042752-199504000-00002","article-title":"Using an objective structured clinical exam to evaluate competency in sport medicine","volume":"5","author":"Mohtadi","year":"1995","journal-title":"Clin J Sport Med"},{"issue":"6","key":"2020060411480919000_i1062-6050-49-3-373-b22","doi-asserted-by":"crossref","first-page":"478","DOI":"10.3109\/0142159X.2011.565828","article-title":"Programmatic assessment: from assessment of learning to assessment for learning","volume":"33","author":"Schuwirth","year":"2011","journal-title":"Med Teach"},{"issue":"2","key":"2020060411480919000_i1062-6050-49-3-373-b23","first-page":"198","article-title":"The feasibility and practicality of employing generalizability theory in performance-based, practical examinations: the SOAT experience [abstract]","volume":"18","author":"Lafave","year":"2008","journal-title":"Clin J Sport Med"},{"issue":"5","key":"2020060411480919000_i1062-6050-49-3-373-b24","doi-asserted-by":"crossref","first-page":"483","DOI":"10.4085\/1062-6050-43.5.483","article-title":"Initial reliability of the Standardized Orthopedic Assessment Tool (SOAT)","volume":"43","author":"Lafave","year":"2008","journal-title":"J Athl Train"},{"issue":"5","key":"2020060411480919000_i1062-6050-49-3-373-b25","doi-asserted-by":"crossref","first-page":"764","DOI":"10.1002\/art.21438","article-title":"ARC Virtual Rheumatology CAL Research Group. Cluster randomized controlled trial of the impact of a computer assisted learning package on the learning of musculoskeletal examination skills by undergraduate medical students","volume":"53","author":"Vivekananda-Schmidt","year":"2005","journal-title":"Arthritis Rheum"},{"issue":"3","key":"2020060411480919000_i1062-6050-49-3-373-b26","first-page":"171","article-title":"An overview of the objective structured clinical exam","volume":"45","author":"Nayer","year":"1993","journal-title":"Physiother Can"},{"issue":"4","key":"2020060411480919000_i1062-6050-49-3-373-b27","doi-asserted-by":"crossref","first-page":"397","DOI":"10.1007\/s10459-006-9050-2","article-title":"Development of a content-valid standardized orthopedic assessment tool (SOAT)","volume":"13","author":"Lafave","year":"2008","journal-title":"Adv Health Sci Educ Theory Pract"},{"issue":"2","key":"2020060411480919000_i1062-6050-49-3-373-b28","doi-asserted-by":"crossref","first-page":"420","DOI":"10.1037\/0033-2909.86.2.420","article-title":"Intraclass correlations: uses in assessing rater reliability","volume":"86","author":"Shrout","year":"1979","journal-title":"Psychol Bull"},{"issue":"1","key":"2020060411480919000_i1062-6050-49-3-373-b29","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1037\/1082-989X.1.1.30","article-title":"Forming inferences about some intraclass correlation coefficients","volume":"1","author":"McGraw","year":"1996","journal-title":"Psychol Method"},{"issue":"5","key":"2020060411480919000_i1062-6050-49-3-373-b30","doi-asserted-by":"crossref","first-page":"314","DOI":"10.1097\/00006199-198109000-00014","article-title":"The intraclass correlation coefficient as a measure of inter-rater reliability of subjective judgments","volume":"30","author":"Armstrong","year":"1981","journal-title":"Nurs Res"},{"issue":"1","key":"2020060411480919000_i1062-6050-49-3-373-b31","first-page":"231","article-title":"Quantifying test-rest reliability using the intraclass correlation coefficient and the SEM","volume":"19","author":"Weir","year":"2005","journal-title":"J Strength Cond Res"},{"issue":"8476","key":"2020060411480919000_i1062-6050-49-3-373-b32","doi-asserted-by":"crossref","first-page":"307","DOI":"10.1016\/S0140-6736(86)90837-8","article-title":"Statistical methods for assessing agreement between two methods of clinical measurement","volume":"1","author":"Bland","year":"1986","journal-title":"Lancet"},{"key":"2020060411480919000_i1062-6050-49-3-373-b33","doi-asserted-by":"crossref","unstructured":"Streiner DL, Norman GR. Health Measurement Scales: A Practical Guide to Their Development and Use. 3rd ed. New York, NY: Oxford University Press;2003.","DOI":"10.1093\/oso\/9780198528470.001.0001"},{"key":"2020060411480919000_i1062-6050-49-3-373-b34","unstructured":"Neufeld VR, Norman GR. Assessing Clinical Competence. New York, NY: Springer;1985."},{"issue":"6","key":"2020060411480919000_i1062-6050-49-3-373-b35","doi-asserted-by":"crossref","first-page":"456","DOI":"10.1037\/h0074469","article-title":"On the loss of reliability in ratings due to coarseness of the scale","volume":"7","author":"Symonds","year":"1924","journal-title":"J Exp Psychol"},{"issue":"2","key":"2020060411480919000_i1062-6050-49-3-373-b36","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1111\/j.2044-8295.1978.tb01647.x","article-title":"Graphical rating scales: how many categories?","volume":"69","author":"McKelvie","year":"1978","journal-title":"Br J Psychol"},{"issue":"6","key":"2020060411480919000_i1062-6050-49-3-373-b37","doi-asserted-by":"crossref","first-page":"476","DOI":"10.7326\/0003-4819-138-6-200303180-00012","article-title":"The mini-CEX: a method for assessing clinical skills","volume":"138","author":"Norcini","year":"2003","journal-title":"Ann Intern Med"},{"issue":"10","key":"2020060411480919000_i1062-6050-49-3-373-b38","doi-asserted-by":"crossref","first-page":"795","DOI":"10.7326\/0003-4819-123-10-199511150-00008","article-title":"The mini-CEX (clinical evaluation exercise): a preliminary investigation","volume":"123","author":"Norcini","year":"1995","journal-title":"Ann Intern Med"},{"key":"2020060411480919000_i1062-6050-49-3-373-b39","unstructured":"Portney LG, Watkins MP. Foundations of Clinical Research: Applications to Practice. 3rd ed. Upper Saddle River, NJ: Pearson\/Prentice Hall;2009."},{"key":"2020060411480919000_i1062-6050-49-3-373-b40","unstructured":"Nunnally JC, Bernstein IH. Psychometric Theory. 3rd ed. New York, NY: McGraw-Hill;1994."},{"issue":"4","key":"2020060411480919000_i1062-6050-49-3-373-b41","doi-asserted-by":"crossref","first-page":"339","DOI":"10.1177\/01466210022031796","article-title":"Performance assessment from the perspective of generalizability theory","volume":"24","author":"Brennan","year":"2000","journal-title":"Appl Psychol Meas"},{"issue":"6","key":"2020060411480919000_i1062-6050-49-3-373-b42","doi-asserted-by":"crossref","first-page":"654","DOI":"10.1080\/01421590310001605642","article-title":"A feasibility study comparing checklists and global rating forms to assess resident performance in clinical skills","volume":"25","author":"Ringsted","year":"2003","journal-title":"Med Teach"},{"issue":"10","key":"2020060411480919000_i1062-6050-49-3-373-b43","doi-asserted-by":"crossref","first-page":"S97","DOI":"10.1097\/00001888-199810000-00058","article-title":"High-stakes examinations: what do we know about measurement? Process-rating forms versus task-specific checklists in an OSCE for medical licensure","volume":"73","author":"Reznick","year":"1998","journal-title":"Acad Med"},{"issue":"3","key":"2020060411480919000_i1062-6050-49-3-373-b44","doi-asserted-by":"crossref","first-page":"250","DOI":"10.1080\/01421590310001002836","article-title":"Validity and the OSCE","volume":"25","author":"Hodges","year":"2003","journal-title":"Med Teach"},{"issue":"2","key":"2020060411480919000_i1062-6050-49-3-373-b45","doi-asserted-by":"crossref","first-page":"100","DOI":"10.1111\/j.1365-2923.1991.tb00035.x","article-title":"The accuracy of standardized patient presentation","volume":"25","author":"Tamblyn","year":"1991","journal-title":"Med Educ"}],"container-title":["Journal of Athletic Training"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/meridian.allenpress.com\/jat\/article-pdf\/49\/3\/373\/1618960\/1062-6050-49_1_12.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/meridian.allenpress.com\/jat\/article-pdf\/49\/3\/373\/1618960\/1062-6050-49_1_12.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,11]],"date-time":"2025-06-11T18:57:27Z","timestamp":1749668247000},"score":1,"resource":{"primary":{"URL":"https:\/\/nata.kglmeridian.com\/view\/journals\/attr\/49\/3\/article-p373.xml"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2014,6,1]]},"references-count":45,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2014,6,1]]}},"URL":"https:\/\/doi.org\/10.4085\/1062-6050-49.1.12","relation":{},"ISSN":["1062-6050"],"issn-type":[{"value":"1062-6050","type":"print"}],"subject":[],"published":{"date-parts":[[2014,6,1]]}}}