{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,14]],"date-time":"2026-11-14T22:18:32Z","timestamp":1794694712943,"version":"4.9.1"},"reference-count":33,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2020,1,9]],"date-time":"2020-01-09T00:00:00Z","timestamp":1578528000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,1,9]],"date-time":"2020-01-09T00:00:00Z","timestamp":1578528000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Sports Med - Open"],"published-print":{"date-parts":[[2020,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Background<\/jats:title><jats:p>Off-field screening tools, such as the Sports Concussion Assessment Tool (SCAT), have been recommended to identify possible concussion following a head impact where the consequences are unclear. However, real-life performance, and diagnostic accuracy of constituent sub-tests, have not been well characterized.<\/jats:p><\/jats:sec><jats:sec><jats:title>Methods<\/jats:title><jats:p>A retrospective cohort study was performed in elite Rugby Union competitions between September 2015 and June 2018. The study population comprised consecutive players identified with a head impact event undergoing off-field assessments with the World Rugby Head Injury Assessment (HIA01) screening tool, an abridged version of the SCAT3. Off-field screening performance was investigated by evaluating real-life removal-from-play outcomes and determining the theoretical diagnostic accuracy of the HIA01 tool, and individual sub-tests, if player-specific baseline or normative sub-test thresholds were strictly applied. The reference standard was clinically diagnosed concussion determined by serial medical assessments.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>One thousand one hundred eighteen head impacts events requiring off-field assessments were identified, resulting in 448 concussions. Real-life removal-from-play decisions demonstrated a sensitivity of 76.8% (95% CI 72.6\u201380.6) and a specificity of 86.6% (95% CI 83.7\u201389.1) for concussion (AUROC 0.82, 95% CI 0.79\u20130.84). Theoretical HIA01 tool performance worsened if pre-season baseline values (sensitivity 89.6%, specificity 33.9%, AUROC 0.62,<jats:italic>p<\/jats:italic>\u2009&lt;\u20090.01) or normative thresholds (sensitivity 80.4%, specificity 69.0%, AUROC 0.75,<jats:italic>p<\/jats:italic>\u2009&lt;\u20090.01) were strictly applied. Symptoms and clinical signs were the HIA01 screening tool sub-tests most predictive for concussion; with immediate memory and tandem gait providing little additional diagnostic value.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusions<\/jats:title><jats:p>These findings support expert recommendations that clinical judgement should be used in the assessment of athletes following head impact events. Substitution of the tandem gait and 5-word immediate memory sub-tests with alternative modes could potentially improve screening tool performance.<\/jats:p><\/jats:sec>","DOI":"10.1186\/s40798-019-0231-y","type":"journal-article","created":{"date-parts":[[2020,1,9]],"date-time":"2020-01-09T16:02:27Z","timestamp":1578585747000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":51,"title":["The performance of the World Rugby Head Injury Assessment Screening Tool: a diagnostic accuracy study"],"prefix":"10.1186","volume":"6","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8532-3500","authenticated-orcid":false,"given":"G. W.","family":"Fuller","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"R.","family":"Tucker","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"L.","family":"Starling","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"E.","family":"Falvey","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"M.","family":"Douglas","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"M.","family":"Raftery","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2020,1,9]]},"reference":[{"issue":"2","key":"231_CR1","doi-asserted-by":"publisher","first-page":"160","DOI":"10.1177\/1941738112462203","volume":"5","author":"HJ McCrea","year":"2013","unstructured":"McCrea HJ, Perrine K, Niogi S, Hartl R. Concussion in sports. Sports Health. 2013 Mar;5(2):160\u20134.","journal-title":"Sports Health"},{"issue":"10","key":"231_CR2","doi-asserted-by":"publisher","first-page":"635","DOI":"10.1136\/bjsports-2018-099079","volume":"52","author":"JS Patricios","year":"2018","unstructured":"Patricios JS, Ardern CL, Hislop MD, Aubry M, Bloomfield P, Broderick C, et al. Implementation of the 2017 Berlin concussion in sport group consensus statement in contact and collision sports: a joint position statement from 11 national and international sports organisations. Br J Sports Med. 2018;52(10):635\u201341.","journal-title":"Br J Sports Med"},{"issue":"11","key":"231_CR3","doi-asserted-by":"crossref","first-page":"888","DOI":"10.1136\/bjsports-2016-097441","volume":"51","author":"J Patricios","year":"2017","unstructured":"Patricios J, Fuller GW, Ellenbogen R, Herring S, Kutcher JS, Loosemore M, et al. What are the critical elements of sideline screening that can be used to establish the diagnosis of concussion? A systematic review. Br J Sports Med. 2017;51(11):888\u201394.","journal-title":"Br J Sports Med"},{"issue":"11","key":"231_CR4","doi-asserted-by":"crossref","first-page":"838","DOI":"10.1136\/bjsports-2017-097699","volume":"51","author":"P McCrory","year":"2017","unstructured":"McCrory P, Meeuwisse W, Dvorak J, Aubry M, Bailes J, Broglio S, et al. Consensus statement on concussion in sport\u2014the 5th international conference on concussion in sport held in Berlin, October 2016. Br J Sports Med. 2017;51(11):838\u201347.","journal-title":"Br J Sports Med"},{"issue":"11","key":"231_CR5","doi-asserted-by":"publisher","first-page":"848","DOI":"10.1136\/bjsports-2016-097466","volume":"51","author":"RDG Echemendia","year":"2017","unstructured":"Echemendia RDG, Putukian M, Leddy J, Makdissi M, Sullivan J, Broglio S, Rafferty M, Schneider K, Kissick J, Meeuwisse W, McCrory P, McCrea M, Dvorak J, Sills A, Aubry M, Engebretsen L, Lossemore M, Fuller G, Kutcher J, Ellenbogen R. The sport concussion assessment tool 5th edition (SCAT5). Br J Sports Med. 2017;51(11):848\u201350.","journal-title":"Br J Sports Med"},{"issue":"1","key":"231_CR6","doi-asserted-by":"publisher","first-page":"64","DOI":"10.1136\/bjsports-2016-096461","volume":"51","author":"CW Fuller","year":"2017","unstructured":"Fuller CW, Fuller GW, Kemp SP, Raftery M. Evaluation of world Rugby's concussion management process: results from Rugby world cup 2015. Br J Sports Med. 2017;51(1):64\u20139.","journal-title":"Br J Sports Med"},{"issue":"8","key":"231_CR7","doi-asserted-by":"publisher","first-page":"529","DOI":"10.1136\/bjsports-2014-093498","volume":"49","author":"GW Fuller","year":"2015","unstructured":"Fuller GW, Kemp SP, Decq P. The international Rugby board (IRB) pitch side concussion assessment trial: a pilot test accuracy study. Br J Sports Med. 2015 Apr;49(8):529\u201335.","journal-title":"Br J Sports Med"},{"key":"231_CR8","unstructured":"SCAT3. Br J Sports Med. 2013 Apr;47(5):259."},{"issue":"4","key":"231_CR9","doi-asserted-by":"publisher","first-page":"347","DOI":"10.1016\/j.jsams.2017.07.025","volume":"21","author":"GW Fuller","year":"2018","unstructured":"Fuller GW, Govind O, Tucker R, Raftery M. Sport concussion assessment tool-third edition normative reference values for professional Rugby union players. J Sci Med Sport. 2018 Apr;21(4):347\u201351.","journal-title":"J Sci Med Sport"},{"issue":"1","key":"231_CR10","doi-asserted-by":"publisher","first-page":"28","DOI":"10.1097\/00042752-200301000-00006","volume":"13","author":"A Collie","year":"2003","unstructured":"Collie A, Maruff P, Makdissi M, McCrory P, McStephen M, Darby D. CogSport: reliability and correlation with conventional cognitive tests used in postconcussion medical evaluations. Clin J Sport Med. 2003 Jan;13(1):28\u201332.","journal-title":"Clin J Sport Med"},{"issue":"3","key":"231_CR11","doi-asserted-by":"publisher","first-page":"287","DOI":"10.1177\/1941738111403122","volume":"3","author":"DR Bell","year":"2011","unstructured":"Bell DR, Guskiewicz KM, Clark MA, Padua DA. Systematic review of the balance error scoring system. Sports Health. 2011 May;3(3):287\u201395.","journal-title":"Sports Health"},{"key":"231_CR12","unstructured":"CSx. 2017 [cited 2017 3rd April 2017]; Available from: http:\/\/csx.co.nz\/"},{"issue":"5","key":"231_CR13","doi-asserted-by":"publisher","first-page":"250","DOI":"10.1136\/bjsports-2013-092313","volume":"47","author":"P McCrory","year":"2013","unstructured":"McCrory P, Meeuwisse WH, Aubry M, Cantu B, Dvorak J, Echemendia RJ, et al. Consensus statement on concussion in sport: the 4th international conference on concussion in sport held in Zurich, November 2012. Br J Sports Med. 2013 Apr;47(5):250\u20138.","journal-title":"Br J Sports Med"},{"key":"231_CR14","volume-title":"Practical statistics for medical research: chapman and hall","author":"DG Altman","year":"1991","unstructured":"Altman DG. Practical statistics for medical research: chapman and hall; 1991."},{"key":"231_CR15","doi-asserted-by":"crossref","unstructured":"Altman DG. Practical statistics for medical research. London: Chapman and Hall; 1991.","DOI":"10.1201\/9780429258589"},{"issue":"3","key":"231_CR16","doi-asserted-by":"publisher","first-page":"837","DOI":"10.2307\/2531595","volume":"44","author":"ER DeLong","year":"1988","unstructured":"DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988 Sep;44(3):837\u201345.","journal-title":"Biometrics."},{"key":"231_CR17","doi-asserted-by":"publisher","DOI":"10.1002\/9781118548387","volume-title":"Applied logistic regression. 3rd ed. ed","author":"DW Hosmer","year":"2013","unstructured":"Hosmer DW, Lemeshow S, Sturdivant RX. Applied logistic regression. 3rd ed. ed. Hoboken: Wiley; 2013."},{"key":"231_CR18","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4757-3462-1","volume-title":"Regression modeling strategies : with applications to linear models, logistic regression, and survival analysis. New York","author":"FE Harrell","year":"2001","unstructured":"Harrell FE. Regression modeling strategies : with applications to linear models, logistic regression, and survival analysis. New York. London: Springer; 2001."},{"key":"231_CR19","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-387-77244-8","volume-title":"Clinical prediction models : a practical approach to development, validation, and updating. New York","author":"EW Steyerberg","year":"2009","unstructured":"Steyerberg EW. Clinical prediction models : a practical approach to development, validation, and updating. New York. London: Springer; 2009."},{"issue":"1","key":"231_CR20","doi-asserted-by":"publisher","first-page":"19","DOI":"10.1198\/000313001300339897","volume":"55","author":"JM Hoenig","year":"2001","unstructured":"Hoenig JM, Heisey DM. The abuse of power. Am Stat. 2001;55(1):19\u201324.","journal-title":"Am Stat"},{"issue":"9","key":"231_CR21","doi-asserted-by":"publisher","first-page":"993","DOI":"10.1016\/j.jclinepi.2010.11.012","volume":"64","author":"DS Courvoisier","year":"2011","unstructured":"Courvoisier DS, Combescure C, Agoritsas T, Gayet-Ageron A, Perneger TV. Performance of logistic regression modeling: beyond the number of events per variable, the role of data structure. J Clin Epidemiol. 2011;64(9):993\u20131000.","journal-title":"J Clin Epidemiol"},{"issue":"20","key":"231_CR22","doi-asserted-by":"publisher","first-page":"1316","DOI":"10.1136\/bjsports-2014-094148","volume":"49","author":"J-S Roy","year":"2015","unstructured":"Roy J-S, Bra\u00ebn C, Leblond J, Desmeules F, Dionne CE, MacDermid JC, et al. Diagnostic accuracy of ultrasonography, MRI and MR arthrography in the characterisation of rotator cuff disorders: a systematic review and meta-analysis. Br J Sports Med. 2015;49(20):1316\u201328.","journal-title":"Br J Sports Med"},{"issue":"3","key":"231_CR23","doi-asserted-by":"publisher","first-page":"338","DOI":"10.1016\/j.annemergmed.2016.12.004","volume":"70","author":"DL Schriger","year":"2017","unstructured":"Schriger DL, Elder JW, Cooper RJ. Structured clinical decision aids are seldom compared with subjective physician judgment, and are seldom superior. Ann Emerg Med. 2017;70(3):338\u201344 e3.","journal-title":"Ann Emerg Med"},{"issue":"1","key":"231_CR24","doi-asserted-by":"crossref","first-page":"2","DOI":"10.1136\/emermed-2018-208324","volume":"36","author":"EJ Weber","year":"2019","unstructured":"Weber EJ, Carlton EW. Side effects of decision rules, or the law of unintended consequences. Emerg Med J. 2019;36(1):2\u20133.","journal-title":"Emerg Med J"},{"key":"231_CR25","doi-asserted-by":"publisher","first-page":"e3999","DOI":"10.1136\/bmj.e3999","volume":"345","author":"S Mallett","year":"2012","unstructured":"Mallett S, Halligan S, Thompson M, Collins GS, Altman DG. Interpreting diagnostic accuracy studies for patient care. BMJ. 2012;345:e3999.","journal-title":"BMJ"},{"issue":"5","key":"231_CR26","doi-asserted-by":"publisher","first-page":"1255","DOI":"10.1007\/s40279-017-0813-0","volume":"48","author":"SP Broglio","year":"2018","unstructured":"Broglio SP, Katz BP, Zhao S, McCrea M, McAllister T, Hoy AR, et al. Test-retest reliability and interpretation of common concussion assessment tools: findings from the NCAA-DoD CARE consortium. Sports Med. 2018;48(5):1255\u201368.","journal-title":"Sports Med"},{"issue":"4","key":"231_CR27","doi-asserted-by":"publisher","DOI":"10.3171\/2016.1.FOCUS15611","volume":"40","author":"AM Yengo-Kahn","year":"2016","unstructured":"Yengo-Kahn AM, Hale AT, Zalneraitis BH, Zuckerman SL, Sills AK, Solomon GS. The sport concussion assessment tool: a systematic review. Neurosurg Focus. 2016;40(4):E6.","journal-title":"Neurosurg Focus"},{"issue":"3","key":"231_CR28","doi-asserted-by":"publisher","first-page":"338","DOI":"10.1111\/j.1651-2227.2006.00180.x","volume":"96","author":"AK Akobeng","year":"2007","unstructured":"Akobeng AK. Understanding diagnostic tests 1: sensitivity, specificity and predictive values. Acta Paediatr. 2007;96(3):338\u201341.","journal-title":"Acta Paediatr"},{"issue":"7","key":"231_CR29","doi-asserted-by":"publisher","first-page":"598","DOI":"10.7326\/0003-4819-137-7-200210010-00011","volume":"137","author":"SA Mulherin","year":"2002","unstructured":"Mulherin SA, Miller WC. Spectrum bias or spectrum effect? Subgroup variation in diagnostic test evaluation. Ann Intern Med. 2002;137(7):598\u2013602.","journal-title":"Ann Intern Med"},{"issue":"4","key":"231_CR30","doi-asserted-by":"publisher","first-page":"447","DOI":"10.1177\/0272989X8500500406","volume":"5","author":"PM Doubilet","year":"1985","unstructured":"Doubilet PM, Cain KC. The superiority of sequential over simultaneous testing. Med Decis Mak. 1985;5(4):447\u201351.","journal-title":"Med Decis Mak"},{"issue":"8","key":"231_CR31","doi-asserted-by":"publisher","first-page":"1335","DOI":"10.1373\/clinchem.2005.048595","volume":"51","author":"AW Rutjes","year":"2005","unstructured":"Rutjes AW, Reitsma JB, Vandenbroucke JP, Glas AS, Bossuyt PM. Case-control and two-gate designs in diagnostic accuracy studies. Clin Chem. 2005;51(8):1335\u201341.","journal-title":"Clin Chem"},{"issue":"1","key":"231_CR32","doi-asserted-by":"publisher","first-page":"W1","DOI":"10.7326\/0003-4819-138-1-200301070-00012-w1","volume":"138","author":"PM Bossuyt","year":"2003","unstructured":"Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig LM, et al. The STARD statement for reporting studies of diagnostic accuracy: explanation and elaboration. Ann Intern Med. 2003;138(1):W1\u201312.","journal-title":"Ann Intern Med"},{"issue":"50","key":"231_CR33","doi-asserted-by":"publisher","first-page":"iii","DOI":"10.3310\/hta11500","volume":"11","author":"AW Rutjes","year":"2007","unstructured":"Rutjes AW, Reitsma JB, Coomarasamy A, Khan KS, Bossuyt PM. Evaluation of diagnostic tests when there is no gold standard. A review of methods. Health Technol Assess. 2007;11(50):iii ix-51.","journal-title":"Health Technol Assess"}],"container-title":["Sports Medicine - Open"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/s40798-019-0231-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1186\/s40798-019-0231-y\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/s40798-019-0231-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,29]],"date-time":"2024-07-29T22:46:47Z","timestamp":1722293207000},"score":1,"resource":{"primary":{"URL":"https:\/\/sportsmedicine-open.springeropen.com\/articles\/10.1186\/s40798-019-0231-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,1,9]]},"references-count":33,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2020,12]]}},"alternative-id":["231"],"URL":"https:\/\/doi.org\/10.1186\/s40798-019-0231-y","relation":{},"ISSN":["2199-1170","2198-9761"],"issn-type":[{"value":"2199-1170","type":"print"},{"value":"2198-9761","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,1,9]]},"assertion":[{"value":"4 June 2019","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"18 December 2019","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"9 January 2020","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Ethical approval was confirmed with the University of Sheffield. All players provided informed consent for the use of HIA data for research purposes. All data were anonymised.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors, G. Fuller, R. Tucker, L. Starling, E. Falvey, M. Douglas, and M. Raftery, declare that they have no competing interests.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"2"}}