{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,15]],"date-time":"2026-10-15T14:45:23Z","timestamp":1792075523338,"version":"4.4.0"},"reference-count":39,"publisher":"American Speech Language Hearing Association","issue":"5S","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Am J Speech Lang Pathol"],"published-print":{"date-parts":[[2026,10,15]]},"abstract":"<jats:sec>\n                    <jats:title>Purpose:<\/jats:title>\n                    <jats:p>This study used generalizability theory (G-theory) to assess the reliability of the Brief Assessment of Transactional Success in aphasia (BATS) and whether reliability can be maintained with more efficient study designs.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Method:<\/jats:title>\n                    <jats:p>G-theory was applied to a data set from BATS test development, where story retells were scored using main concept analysis. Linear mixed-effects models were used to estimate the proportion of variability in scores attributable to storyteller (a person with aphasia [PWA] or familiar\/unfamiliar conversation partner [CP]), rater (human or automated), and stimuli. Decision studies (D-studies) were conducted to determine the number of stimuli needed for acceptable reliability. Score distributions and correlations were examined to identify stimuli that disproportionately contributed to measurement error.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results:<\/jats:title>\n                    <jats:p>\n                      Differences in performance between PWA contributed most to variability in scores, whereas rater contributed minimally. Stimuli contributed substantially, primarily through interactions with PWA and storyteller. Two stimuli (\n                      <jats:italic toggle=\"yes\">Bicycle Boy<\/jats:italic>\n                      and\n                      <jats:italic toggle=\"yes\">Ferguson<\/jats:italic>\n                      ) emerged as the least and most challenging; when excluded, stimuli were no longer a significant source of variability. D-studies showed that reliable results could be obtained with three to four stimuli, or as few as two when\n                      <jats:italic toggle=\"yes\">Bicycle Boy<\/jats:italic>\n                      and\n                      <jats:italic toggle=\"yes\">Ferguson<\/jats:italic>\n                      were excluded. CP familiarity contributed to variability in scores via its interaction with PWA.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions:<\/jats:title>\n                    <jats:p>The BATS demonstrated high reliability despite multiple sources of variability in scores. Results support the reliability of automated main concept scoring and show that stimuli-driven error can be controlled with a limited number of stimuli. These results offer guidance for designing efficient, reliable discourse studies.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1044\/2026_ajslp-25-00398","type":"journal-article","created":{"date-parts":[[2026,10,12]],"date-time":"2026-10-12T17:24:28Z","timestamp":1791825868000},"page":"2742-2753","source":"Crossref","is-referenced-by-count":0,"title":["A Study of Variability in Brief Assessment of Transactional Success in Aphasia Story Retelling Performance Using Generalizability Theory"],"prefix":"10.1044","volume":"35","author":[{"ORCID":"https:\/\/orcid.org\/0009-0006-9376-879X","authenticated-orcid":false,"given":"Alexis","family":"Doyle-Connolly","sequence":"first","affiliation":[{"name":"Department of Mathematics and Statistics, University of Massachusetts Amherst"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jacquie","family":"Kurland","sequence":"additional","affiliation":[{"name":"Department of Speech, Language, and Hearing Sciences, University of Massachusetts Amherst"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Anna","family":"Liu","sequence":"additional","affiliation":[{"name":"Department of Mathematics and Statistics, University of Massachusetts Amherst"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pauline","family":"Stokes","sequence":"additional","affiliation":[{"name":"Department of Speech, Language, and Hearing Sciences, University of Massachusetts Amherst"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1310","reference":[{"key":"e_1_3_2_2_1","doi-asserted-by":"publisher","DOI":"10.1080\/02687038.2011.577204"},{"key":"e_1_3_2_3_1","doi-asserted-by":"publisher","DOI":"10.1080\/02687030050127685"},{"key":"e_1_3_2_4_1","doi-asserted-by":"publisher","DOI":"10.18637\/jss.v067.i01"},{"key":"e_1_3_2_5_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4757-3456-0"},{"key":"e_1_3_2_6_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jsp.2013.11.008"},{"key":"e_1_3_2_7_1","doi-asserted-by":"publisher","DOI":"10.3109\/02699206.2016.1145740"},{"key":"e_1_3_2_8_1","doi-asserted-by":"publisher","DOI":"10.1111\/1460-6984.12957"},{"key":"e_1_3_2_9_1","volume-title":"Novel web applications to facilitate discourse analysis in everyday clinical practice [Paper presentation]","author":"Cavanaugh R.","year":"2022","unstructured":"Cavanaugh, R., Dalton, S., & Richardson, J. (2022, November). Novel web applications to facilitate discourse analysis in everyday clinical practice [Paper presentation]. American Speech-Language-Hearing Association Annual Convention, New Orleans, Louisiana."},{"key":"e_1_3_2_10_1","unstructured":"Cavanaugh R. Richardson J. D. & Dalton S. G. (2021). mainConcept: An open-source web app for scoring main concept analysis (Version 0.0.1.0000) [R package]. GitHub. https:\/\/github.com\/aphasia-apps\/mainConcept"},{"key":"e_1_3_2_11_1","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511620539"},{"key":"e_1_3_2_12_1","volume-title":"The dependability of behavioral measurements: Theory of generalizability for scores and profiles","author":"Cronbach L. J.","year":"1972","unstructured":"Cronbach, L. J., Gleser, G. C., Nanda, H., & Rajaratnam, N. (1972). The dependability of behavioral measurements: Theory of generalizability for scores and profiles. Wiley."},{"key":"e_1_3_2_13_1","doi-asserted-by":"publisher","DOI":"10.1177\/0013164404266386"},{"key":"e_1_3_2_14_1","doi-asserted-by":"publisher","DOI":"10.1111\/1460-6984.12528"},{"key":"e_1_3_2_15_1","doi-asserted-by":"publisher","DOI":"10.1044\/2018_AJSLP-17-0166"},{"key":"e_1_3_2_16_1","doi-asserted-by":"publisher","DOI":"10.3389\/fcomm.2021.574051"},{"key":"e_1_3_2_17_1","doi-asserted-by":"publisher","DOI":"10.1080\/02687038.2011.606974"},{"key":"e_1_3_2_18_1","doi-asserted-by":"publisher","DOI":"10.1044\/2021_JSLHR-20-00586"},{"key":"e_1_3_2_19_1","doi-asserted-by":"publisher","DOI":"10.1044\/2020_JSLHR-19-00118"},{"key":"e_1_3_2_20_1","doi-asserted-by":"publisher","DOI":"10.1177\/0011000099273003"},{"key":"e_1_3_2_21_1","doi-asserted-by":"publisher","DOI":"10.1207\/s15327973rlsi2803_1"},{"key":"e_1_3_2_22_1","doi-asserted-by":"publisher","DOI":"10.2307\/800595"},{"key":"e_1_3_2_23_1","author":"Kertesz A.","year":"2007","unstructured":"Kertesz, A. (2007). Western Aphasia Battery\u2013Revised. The Psychological Corporation.","journal-title":"Western Aphasia Battery\u2013Revised"},{"key":"e_1_3_2_24_1","doi-asserted-by":"publisher","DOI":"10.1080\/02687038.2021.1988046"},{"key":"e_1_3_2_25_1","doi-asserted-by":"publisher","DOI":"10.1080\/02687038.2024.2351029"},{"key":"e_1_3_2_26_1","doi-asserted-by":"publisher","DOI":"10.1080\/02687038.2017.1398808"},{"key":"e_1_3_2_27_1","doi-asserted-by":"publisher","DOI":"10.1044\/2025_AJSLP-24-00400"},{"key":"e_1_3_2_28_1","article-title":"Development and validation of the breast cancer scale QLICP-BR V2.0 based on classical test theory and generalizability theory","volume":"12","author":"Li F.","year":"2022","unstructured":"Li, F., Zhou, J., Wan, C., Yang, Z., Liang, Q., Li, W., & Chen, H. (2022). Development and validation of the breast cancer scale QLICP-BR V2.0 based on classical test theory and generalizability theory. Frontiers in Oncology, 12, Article 915103. https:\/\/doi.org\/10.3389\/fonc.2022.915103","journal-title":"Frontiers in Oncology"},{"key":"e_1_3_2_29_1","unstructured":"R Core Team. (2024). R: A language and environment for statistical computing (Version 4.3.3) [Computer software]. R Foundation for Statistical Computing. https:\/\/www.R-project.org\/"},{"key":"e_1_3_2_30_1","doi-asserted-by":"publisher","DOI":"10.1080\/02687040143000636"},{"key":"e_1_3_2_31_1","doi-asserted-by":"publisher","DOI":"10.1080\/1368282021000060308"},{"key":"e_1_3_2_32_1","doi-asserted-by":"publisher","DOI":"10.1080\/02687038.2015.1057891"},{"key":"e_1_3_2_33_1","doi-asserted-by":"publisher","DOI":"10.1080\/02687038.2018.1561417"},{"key":"e_1_3_2_34_1","doi-asserted-by":"publisher","DOI":"10.1037\/0003-066X.44.6.922"},{"key":"e_1_3_2_35_1","doi-asserted-by":"publisher","DOI":"10.1044\/2019_AJSLP-18-0265"},{"key":"e_1_3_2_36_1","doi-asserted-by":"publisher","DOI":"10.1080\/23273798.2020.1862258"},{"key":"e_1_3_2_37_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4612-3262-9_8"},{"key":"e_1_3_2_38_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1745-3984.1981.tb00839.x"},{"key":"e_1_3_2_39_1","doi-asserted-by":"publisher","DOI":"10.1002\/0470013192.bsa703"},{"key":"e_1_3_2_40_1","doi-asserted-by":"publisher","DOI":"10.1080\/13825585.2013.794894"}],"container-title":["American Journal of Speech-Language Pathology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/pubs.asha.org\/doi\/pdf\/10.1044\/2026_AJSLP-25-00398","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,15]],"date-time":"2026-10-15T14:01:26Z","timestamp":1792072886000},"score":1,"resource":{"primary":{"URL":"https:\/\/pubs.asha.org\/doi\/10.1044\/2026_AJSLP-25-00398"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,10,15]]},"references-count":39,"journal-issue":{"issue":"5S","published-print":{"date-parts":[[2026,10,15]]}},"alternative-id":["10.1044\/2026_AJSLP-25-00398"],"URL":"https:\/\/doi.org\/10.1044\/2026_ajslp-25-00398","relation":{},"ISSN":["1058-0360","1558-9110"],"issn-type":[{"value":"1058-0360","type":"print"},{"value":"1558-9110","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,10,15]]}}}